Terminal

ABSTRACT

A terminal is disclosed. The terminal according to an embodiment of the present invention comprises: an output unit for outputting a notification; a storage unit for storing a database; a control unit for controlling the outputting of the notification; and an artificial intelligence unit for acquiring information regarding a user&#39;s context, and outputting a notification when the user&#39;s context corresponds to information included in the database, wherein the database includes at least one of a user&#39;s personal database, a standard activity database, and an accident type database.

TECHNICAL FIELD

The present disclosure relates to a terminal capable of providing aguide to a user's behavior using various databases.

BACKGROUND ART

Artificial intelligence refers to one field of computer engineering andinformation technology of studying a method for making a computer think,learn, and do self-improvement, which is achieved based on humanintelligence, and means that a computer emulates an intelligent behaviorof the human.

AI is largely related directly and indirectly to other fields of acomputer science rather than existing itself. In particular, AI elementshave been modernly introduced in various fields of informationtechnology, and there has been an active attempt to use AI to overcomeproblems of the fields.

Research has been actively conducted into context awareness technologyof recognizing a situation of a user and providing information desiredby the user in a desired form using conventional AI.

The aforementioned context awareness technology has been developed, andsimultaneously, demands for mobile terminals for performing functionssuitable for a situation of a user have increased.

Meanwhile, it is necessary for a user to know user's past behaviors orhabits of repeating certain behaviors. However, the user may notremember them or may not even recognize their habits.

Therefore, there is an increasing need for reminding a user of user'sbehaviors or habits using a context awareness function of artificialintelligence.

DISCLOSURE Technical Problem

An object of the present disclosure devised to solve the problem lies ina terminal capable of providing a guide to a user's behavior usingvarious databases.

Technical Solution

A terminal according to an embodiment may include an output interfaceconfigured to output a notification, a memory configured to store adatabase, a controller configured to control output of the notification,and an artificial intelligence interface configured to acquireinformation on a situation of a user and output a notification when thesituation of the user corresponds to information included in thedatabase. The database includes at least one of a personal database ofthe user, a standard behavior database or an accident type database.

In this case, the standard behavior database may include information ona behavior of a driver while driving, and the artificial intelligenceinterface may acquire information on a behavior of the user and outputsthe notification when the behavior of the user corresponds to thebehavior of the driver included in the standard behavior database.

In this case, the standard behavior database may include information ona position and information on a behavior of a driver at the position,and the artificial intelligence interface may output the notificationwhen the user passes the position and the behavior of the usercorresponds to the behavior of the driver at the position.

The standard behavior database may include information on an incorrectbehavior of a driver while driving, the memory may include informationon a correct behavior corresponding to the incorrect behavior, and theartificial intelligence interface may output the correct behavior whenthe behavior of the user corresponds to the incorrect behavior.

The personal database of the user may include information on a behaviorpattern acquired from a past behavior of the user, and the artificialintelligence interface may acquire information on the behavior of theuser and determine whether the notification is output or an intensity ofthe notification based on whether the behavior of the user correspondsto the behavior pattern.

The artificial intelligence interface may acquire and store a behaviorpattern of a user corresponding to the standard behavior database basedon a behavior of the user and the standard behavior database, and outputthe notification when the behavior of the user corresponds to thebehavior pattern.

The accident type database may include information on a position andinformation on an accident type at the position, and the artificialintelligence interface may output a notification corresponding to theaccident type at the position when the user passes the position.

In this case, the artificial intelligence interface may acquireinformation on a driving state of the user, and output a notificationcorresponding to the accident type based on the driving state of theuser and the accident type when the user passes the position.

The artificial intelligence interface may acquire a driving habitincluding at least one of information on a traffic regulation violationof the user or information on route deviation of the user, store thedriving habit and position information corresponding to the drivinghabit in the personal database, and output a notification related to thedriving habit at a position where the driving habit appears.

In this case, the artificial intelligence interface may output anotification for preventing the traffic regulation violation when theuser reaches a position where the traffic regulation violation appears,based on the information on the traffic regulation violation andposition information corresponding to the traffic regulation violation.

The artificial intelligence interface may output a notification forpreventing the route deviation when the user reaches a position wherethe route deviation appears, based on the information on the routedeviation and position information corresponding to the information onthe route deviation.

The artificial intelligence interface may acquire information on amovement route to be used by the user based on a past movement route ofthe user and current movement of the user and provide a guide to a newroute based on route information of the movement route to be used by theuser, and the route information may include at least one of a trafficcondition of the route or a route faster than a route used by the userin the past.

The personal database may include a consumption list of the user, andthe artificial intelligence interface may acquire a consumption patternof the user based on the consumption list of the user, and output anotification indicating a consumption state of the user based on theconsumption pattern and the consumption list.

The personal database may include a list of items of the user, and theartificial intelligence interface may acquire information on an item,which needs to be purchased, based on at least one of a purchase time, apurchase frequency or a distribution period of an item included in thelist of items, and output a notification for purchasing the item in thevicinity of a place where the item, which needs to be purchased, iscapable of being acquired.

The standard behavior database may include information on a person'slife pattern, and the artificial intelligence interface may acquireinformation on a life pattern of the user, and output a warningnotification based on information on the life pattern of the user andinformation on the person's life pattern.

DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating a mobile terminal according tothe present disclosure.

FIGS. 1b and 1c are conceptual views showing an examples of a mobileterminal according to the present disclosure when viewed from differentdirections.

FIG. 2 is a view illustrating a method of operating a terminal accordingto an embodiment of the present disclosure.

FIG. 3 is a view illustrating a driver guide system including a terminalaccording to an embodiment of the present disclosure.

FIGS. 4 to 5 are views illustrating a method of outputting anotification according to a user's behavior according to an embodimentof the present disclosure.

FIG. 6 is a view illustrating a method of determining whether anotification is output and the intensity of the notification inconsideration of the behavior pattern of a user according to anembodiment of the present disclosure.

FIG. 7 is a view illustrating a method of outputting a notificationusing a behavior pattern of a user corresponding to a standard behaviordatabase according to an embodiment of the present disclosure.

FIGS. 8 to 9 are views illustrating a method of outputting anotification based on an accident type database according to anembodiment of the present disclosure.

FIG. 10 is a view illustrating a method of outputting a notificationcorresponding to an accident type based on an accident type database andinformation on a driving state of a user according to an embodiment ofthe present disclosure.

FIG. 11 is a view illustrating a method of guiding a user based on adriving habit of a user according to an embodiment of the presentdisclosure.

FIGS. 12 to 15 are views illustrating a method of guiding a route basedon a route pattern of a user according to an embodiment of the presentdisclosure.

FIGS. 16 to 17 are views illustrating a method of outputting anotification based on a consumption pattern of a user according to anembodiment of the present disclosure.

FIG. 18 is a view illustrating a method of providing a guide to an itemto be purchased according to an embodiment of the present disclosure.

FIG. 19 is a view illustrating a method of outputting a life patternrecommended to a user based on information on a preferable life patternof a person according to an embodiment of the present disclosure.

BEST MODE

Description will now be given in detail according to exemplaryembodiments disclosed herein, with reference to the accompanyingdrawings. For the sake of brief description with reference to thedrawings, the same or equivalent components may be provided with thesame reference numbers, and description thereof will not be repeated. Ingeneral, a suffix such as “module” and “unit” may be used to refer toelements or components. Use of such a suffix herein is merely intendedto facilitate description of the specification, and the suffix itself isnot intended to give any special meaning or function. In the presentdisclosure, that which is well-known to one of ordinary skill in therelevant art has generally been omitted for the sake of brevity. Theaccompanying drawings are used to help easily understand varioustechnical features and it should be understood that the embodimentspresented herein are not limited by the accompanying drawings. As such,the present disclosure should be construed to extend to any alterations,equivalents and substitutes in addition to those which are particularlyset out in the accompanying drawings.

It will be understood that although the terms first, second, etc. may beused herein to describe various elements, these elements should not belimited by these terms. These terms are generally only used todistinguish one element from another.

It will be understood that when an element is referred to as being“connected with” another element, the element can be connected with theother element or intervening elements may also be present. In contrast,when an element is referred to as being “directly connected with”another element, there are no intervening elements present.

A singular representation may include a plural representation unless itrepresents a definitely different meaning from the context. Terms suchas “include” or “has” are used herein and should be understood that theyare intended to indicate an existence of several components, functionsor steps, disclosed in the specification, and it is also understood thatgreater or fewer components, functions, or steps may likewise beutilized.

Mobile terminals presented herein may be implemented using a variety ofdifferent types of terminals. Examples of such terminals includecellular phones, smart phones, user equipment, laptop computers, digitalbroadcast terminals, personal digital assistants (PDAs), portablemultimedia players (PMPs), navigators, portable computers (PCs), slatePCs, tablet PCs, ultra books, wearable devices (for example, smartwatches, smart glasses, head mounted displays (HMDs)), and the like.

By way of non-limiting example only, further description will be madewith reference to particular types of mobile terminals. However, suchteachings apply equally to other types of terminals, such as those typesnoted above. In addition, these teachings may also be applied tostationary terminals such as digital TV, desktop computers, and thelike.

Reference is now made to FIGS. 1A-1C, where FIG. 1A is a block diagramof a mobile terminal in accordance with the present disclosure, andFIGS. 1B and 1C are conceptual views of one example of the mobileterminal, viewed from different directions.

The mobile terminal 100 is shown having components such as a wirelesscommunication unit 110, an input interface 120, a sensing unit 140, anoutput interface 150, an interface 160, a memory 170, a controller 180,and a power supply unit 190. It is understood that implementing all ofthe illustrated components is not a requirement, and that greater orfewer components may alternatively be implemented.

Referring now to FIG. 1A, the mobile terminal 100 is shown havingwireless communication unit 110 configured with several commonlyimplemented components. For instance, the wireless communication unit110 typically includes one or more components which permit wirelesscommunication between the mobile terminal 100 and a wirelesscommunication system or network within which the mobile terminal islocated.

The wireless communication unit 110 typically includes one or moremodules which permit communications such as wireless communicationsbetween the mobile terminal 100 and a wireless communication system,communications between the mobile terminal 100 and another mobileterminal, communications between the mobile terminal 100 and an externalserver. Further, the wireless communication unit 110 typically includesone or more modules which connect the mobile terminal 100 to one or morenetworks.

To facilitate such communications, the wireless communication unit 110includes one or more of a broadcast receiving module 111, a mobilecommunication module 112, a wireless Internet module 113, a short-rangecommunication module 114, and a location information module 115.

The input interface 120 includes a camera 121 for obtaining images orvideo, a microphone 122, which is one type of audio input device forinputting an audio signal, and a user input interface 123 (for example,a touch key, a push key, a mechanical key, a soft key, and the like) forallowing a user to input information. Data (for example, audio, video,image, and the like) is obtained by the input interface 120 and may beanalyzed and processed by controller 180 according to device parameters,user commands, and combinations thereof.

An artificial intelligence part 130 may process information based onartificial intelligence technology and may include one or more modulesfor performing at least one of information learning, informationinference, information perception, or natural language processing.

The artificial intelligence part 130 may perform at least one oflearning, inference, or processing on a massive amount of information(big data) such as information stored in a mobile terminal, informationon an environment around the mobile terminal, or information stored inan external memory that is capable of communicating, using machinelearning technology. The artificial intelligence part 130 may predict(or infer) at least one executable operation of a mobile terminal usinglearned information using the machine learning technology, and maycontrol the mobile terminal to perform an operation with the highestrealistic possibility among the at least one predicted operations.

The machine learning technology is technology of collecting and learninga massive amount of information and determining and predictinginformation based on the learned information, using at least onealgorithm. The information learning is an operation of determining theproperty, rule, and determination reference of information, quantifyinga relationship between information items, and predicting new data usingthe quantified pattern.

The machine learning technology may use an algorithm based onstatistics, for example, a decision tree that uses a tree structure formas a predictive model, a neural network that emulates a structure andfunction of a neural network structure of a living organism, geneticprogramming based on an evolution algorithm of a living organism,clustering for distributing observed examples to subsets that are each acluster, or a Monter carlo method of calculating a function value usingprobability through a random number that is arbitrarily extracted.

Deep learning technology that is one field of machine learningtechnology performs at least one of learning, determination, orprocessing of information using the neural network algorithm. The neuralnetwork may have a structure for connection between layers andtransferring data between layers. The deep learning technology may learna massive amount of information through a neural network using a graphicprocessing unit (GPU) that is optimized for parallel arithmetic.

The artificial intelligence part 130 may collect (percept, monitor,extract, detect, or receive) a signal, data, information, or the like,which is input or output from components of the mobile terminal in orderto collect a massive amount of information for applying the machinelearning technology. The artificial intelligence part 130 may collect(percept, monitor, extract, detect, or receive) data, information, orthe like, which is stored in an external memory (e.g., a cloud server ora cloud server) connected via communication. In more detail, informationcollection may be understood to be a term including an operation ofdetecting information through a sensor, extracting information stored inthe memory 170, or receiving information from the external memory viacommunication.

The artificial intelligence part 130 may detect information in themobile terminal, information on an environment around the mobileterminal, and user information through the sensing unit 140. Theartificial intelligence part 130 may receive broadcast signal, broadcastrelated information, wireless signal, and/or wireless data through thewireless communication unit 110. The artificial intelligence part 130may receive image information (or a signal), audio information (or asignal), and data from an input interface, or information input from auser.

The artificial intelligence part 130 may collect a massive amount ofinformation in real time on a background, may learn the information, andmay store information (e.g., a knowledge graph, a command policy, apersonalized database, a chat engine, or the like) processed in a properform, in the memory 170.

When an operation of the mobile terminal is predicted based on learnedinformation using the machine learning technology, the artificialintelligence part 130 may control components of the mobile terminal ormay transfer a control command for executing the predicted operation tothe controller 180 in order to perform the predicted operation. Thecontroller 180 may control the mobile terminal based on the controlcommand and may perform the predicted operation.

When a specific operation is performed, the artificial intelligence part130 may analyze history information indicating that the specificinformation is performed and may update existing learned informationbased on the analyzed information through the machine learningtechnology. Thus, the artificial intelligence part 130 may increase theaccuracy of predicting information.

In the specification, the artificial intelligence part 130 and thecontroller 180 may be understood to be the same component. In this case,the function perform by the controller 180 described in thespecification may be expressed as being performed by the artificialintelligence part 130, and the controller 180 may also be referred to asthe artificial intelligence part 130, or conversely, the artificialintelligence part 130 may also be referred to as the controller 180.

In contrast, in the specification, the artificial intelligence part 130and the controller 180 may be understood to be separate components. Inthis case, the artificial intelligence part 130 and the controller 180may variously perform control on the mobile terminal by exchanging datawith each other. The controller 180 may perform at least one function onthe mobile terminal or may control at least one of components of themobile terminal, based on the result derived from the artificialintelligence part 130. In addition, the artificial intelligence part 130may also be operated under control of the controller 180.

The sensing unit 140 is typically implemented using one or more sensorsconfigured to sense internal information of the mobile terminal, thesurrounding environment of the mobile terminal, user information, andthe like. For example, in FIG. 1A, the sensing unit 140 is shown havinga proximity sensor 141 and an illumination sensor 142.

If desired, the sensing unit 140 may alternatively or additionallyinclude other types of sensors or devices, such as a touch sensor, anacceleration sensor, a magnetic sensor, a G-sensor, a gyroscope sensor,a motion sensor, an RGB sensor, an infrared (IR) sensor, a finger scansensor, a ultrasonic sensor, an optical sensor (for example, camera121), a microphone 122, a battery gauge, an environment sensor (forexample, a barometer, a hygrometer, a thermometer, a radiation detectionsensor, a thermal sensor, and a gas sensor, among others), and achemical sensor (for example, an electronic nose, a health care sensor,a biometric sensor, and the like), to name a few. The mobile terminal100 may be configured to utilize information obtained from sensing unit140, and in particular, information obtained from one or more sensors ofthe sensing unit 140, and combinations thereof.

The output interface 150 is typically configured to output various typesof information, such as audio, video, tactile output, and the like. Theoutput interface 150 is shown having a display 151, an audio outputmodule 152, a haptic module 153, and an optical output module 154. Thedisplay 151 may have an inter-layered structure or an integratedstructure with a touch sensor in order to facilitate a touch screen. Thetouch screen may provide an output interface between the mobile terminal100 and a user, as well as function as the user input interface 123which provides an input interface between the mobile terminal 100 andthe user.

The interface 160 serves as an interface with various types of externaldevices that can be coupled to the mobile terminal 100. The interface160, for example, may include any of wired or wireless ports, externalpower supply ports, wired or wireless data ports, memory card ports,ports for connecting a device having an identification module, audioinput/output (I/O) ports, video I/O ports, earphone ports, and the like.In some cases, the mobile terminal 100 may perform assorted controlfunctions associated with a connected external device, in response tothe external device being connected to the interface 160.

The memory 170 is typically implemented to store data to support variousfunctions or features of the mobile terminal 100. For instance, thememory 170 may be configured to store application programs executed inthe mobile terminal 100, data or instructions for operations of themobile terminal 100, and the like. Some of these application programsmay be downloaded from an external server via wireless communication.Other application programs may be installed within the mobile terminal100 at time of manufacturing or shipping, which is typically the casefor basic functions of the mobile terminal 100 (for example, receiving acall, placing a call, receiving a message, sending a message, and thelike). It is common for application programs to be stored in the memory170, installed in the mobile terminal 100, and executed by thecontroller 180 to perform an operation (or function) for the mobileterminal 100.

The controller 180 typically functions to control overall operation ofthe mobile terminal 100, in addition to the operations associated withthe application programs. The controller 180 may provide or processinformation or functions appropriate for a user by processing signals,data, information and the like, which are input or output by theabove-described components, or activating application programs stored inthe memory 170.

In addition, the controller 180 controls at least some of the componentsillustrated in FIG. 1A, in order to execute the application programstored in the memory 170. Further, the controller 180 may operate acombination of at least two of the components included in the mobileterminal 100, in order to execute the application program.

The power supply unit 190 can be configured to receive external power orprovide internal power in order to supply appropriate power required foroperating elements and components included in the mobile terminal 100.The power supply unit 190 may include a battery, and the battery may beconfigured to be embedded in the terminal body, or configured to bedetachable from the terminal body.

Hereinafter, prior to description of various embodiments implemented viathe above-described mobile terminal 100, the above-described componentswill be described in greater detail with reference to FIG. 1A.

Regarding the wireless communication unit 110, the broadcast receivingmodule 111 is typically configured to receive a broadcast signal and/orbroadcast associated information from an external broadcast managingentity via a broadcast channel. The broadcast channel may include asatellite channel, a terrestrial channel, or both. In some embodiments,two or more broadcast receiving modules 111 may be utilized tofacilitate simultaneously receiving of two or more broadcast channels,or to support switching among broadcast channels.

The broadcast managing entity may be a server which generates andtransmits a broadcast signal and/or broadcast associated information, ora server which receives a pre-generated broadcast signal and/orbroadcast associated information, and sends such items to the mobileterminal. The broadcast signal may be implemented using any of a TVbroadcast signal, a radio broadcast signal, a data broadcast signal, andcombinations thereof, among others. The broadcast signal in some casesmay further include a data broadcast signal combined with a TV or radiobroadcast signal.

The broadcast signal may be encoded according to any of a variety oftechnical standards or broadcasting methods (for example, InternationalOrganization for Standardization (ISO), International ElectrotechnicalCommission (IEC), Digital Video Broadcast (DVB), Advanced TelevisionSystems Committee (ATSC), and the like) for transmission and receptionof digital broadcast signals. The broadcast receiving module 111 canreceive the digital broadcast signals using a method appropriate for thetransmission method utilized.

Examples of broadcast associated information may include informationassociated with a broadcast channel, a broadcast program, a broadcastevent, a broadcast service provider, or the like. The broadcastassociated information may also be provided via a mobile communicationnetwork, and in this case, received by the mobile communication module112.

The broadcast associated information may be implemented in variousformats. For instance, broadcast associated information may include anElectronic Program Guide (EPG) of Digital Multimedia Broadcasting (DMB),an Electronic Service Guide (ESG) of Digital Video Broadcast-Handheld(DVB-H), and the like. Broadcast signals and/or broadcast associatedinformation received via the broadcast receiving module 111 may bestored in a suitable device, such as a memory 170.

The mobile communication module 112 can transmit and/or receive wirelesssignals to and from one or more network entities. Typical examples of anetwork entity include a base station, an external mobile terminal, aserver, and the like. Such network entities form part of a mobilecommunication network, which is constructed according to technicalstandards or communication methods for mobile communications (forexample, Global System for Mobile Communication (GSM), Code DivisionMulti Access (CDMA), CDMA2000(Code Division Multi Access 2000),EV-DO(Enhanced Voice-Data Optimized or Enhanced Voice-Data Only),Wideband CDMA (WCDMA), High Speed Downlink Packet access (HSDPA),HSUPA(High Speed Uplink Packet Access), Long Term Evolution (LTE) ,LTE-A(Long Term Evolution-Advanced), and the like).

Examples of wireless signals transmitted and/or received via the mobilecommunication module 112 include audio call signals, video (telephony)call signals, or various formats of data to support communication oftext and multimedia messages.

The wireless Internet module 113 is configured to facilitate wirelessInternet access. This module may be internally or externally coupled tothe mobile terminal 100. The wireless Internet module 113 may transmitand/or receive wireless signals via communication networks according towireless Internet technologies.

Examples of such wireless Internet access include Wireless LAN (WLAN),Wireless Fidelity (Wi-Fi), Wi-Fi Direct, Digital Living Network Alliance(DLNA), Wireless Broadband (WiBro), Worldwide Interoperability forMicrowave Access (WiMAX), High Speed Downlink Packet Access (HSDPA),HSUPA(High Speed Uplink Packet Access), Long Term Evolution (LTE),LTE-A(Long Term Evolution-Advanced), and the like. The wireless Internetmodule 113 may transmit/receive data according to one or more of suchwireless Internet technologies, and other Internet technologies as well.

In some embodiments, when the wireless Internet access is implementedaccording to, for example, WiBro, HSDPA,HSUPA, GSM, CDMA, WCDMA, LTE,LTE-A and the like, as part of a mobile communication network, thewireless Internet module 113 performs such wireless Internet access. Assuch, the Internet module 113 may cooperate with, or function as, themobile communication module 112.

The short-range communication module 114 is configured to facilitateshort-range communications. Suitable technologies for implementing suchshort-range communications include BLUETOOTH™, Radio FrequencyIDentification (RFID), Infrared Data Association (IrDA), Ultra-WideBand(UWB), ZigBee, Near Field Communication (NFC), Wireless-Fidelity(Wi-Fi), Wi-Fi Direct, Wireless USB(Wireless Universal Serial Bus), andthe like. The short-range communication module 114 in general supportswireless communications between the mobile terminal 100 and a wirelesscommunication system, communications between the mobile terminal 100 andanother mobile terminal 100, or communications between the mobileterminal and a network where another mobile terminal 100 (or an externalserver) is located, via wireless area networks. One example of thewireless area networks is a wireless personal area networks.

In some embodiments, another mobile terminal (which may be configuredsimilarly to the mobile terminal 100) may be a wearable device, forexample, a smart watch, a smart glass or a head mounted display (HMD),which is able to exchange data with the mobile terminal 100 (orotherwise cooperate with the mobile terminal 100). The short-rangecommunication module 114 may sense or recognize the wearable device, andpermit communication between the wearable device and the mobile terminal100. In addition, when the sensed wearable device is a device which isauthenticated to communicate with the mobile terminal 100, thecontroller 180, for example, may cause transmission of data processed inthe mobile terminal 100 to the wearable device via the short-rangecommunication module 114. Hence, a user of the wearable device may usethe data processed in the mobile terminal 100 on the wearable device.For example, when a call is received in the mobile terminal 100, theuser may answer the call using the wearable device. Also, when a messageis received in the mobile terminal 100, the user can check the receivedmessage using the wearable device.

The location information module 115 is generally configured to detect,calculate, derive or otherwise identify a position of the mobileterminal. As an example, the location information module 115 includes aGlobal Position System (GPS) module, a Wi-Fi module, or both. Ifdesired, the location information module 115 may alternatively oradditionally function with any of the other modules of the wirelesscommunication unit 110 to obtain data related to the position of themobile terminal.

As one example, when the mobile terminal uses a GPS module, a positionof the mobile terminal may be acquired using a signal sent from a GPSsatellite. As another example, when the mobile terminal uses the Wi-Fimodule, a position of the mobile terminal can be acquired based oninformation related to a wireless access point (AP) which transmits orreceives a wireless signal to or from the Wi-Fi module.

The input interface 120 may be configured to permit various types ofinput to the mobile terminal 120. Examples of such input include audio,image, video, data, and user input. Image and video input is oftenobtained using one or more cameras 121. Such cameras 121 may processimage frames of still pictures or video obtained by image sensors in avideo or image capture mode. The processed image frames can be displayedon the display 151 or stored in memory 170. In some cases, the cameras121 may be arranged in a matrix configuration to permit a plurality ofimages having various angles or focal points to be input to the mobileterminal 100. As another example, the cameras 121 may be located in astereoscopic arrangement to acquire left and right images forimplementing a stereoscopic image.

The microphone 122 is generally implemented to permit audio input to themobile terminal 100. The audio input can be processed in various mannersaccording to a function being executed in the mobile terminal 100. Ifdesired, the microphone 122 may include assorted noise removingalgorithms to remove unwanted noise generated in the course of receivingthe external audio.

The user input interface 123 is a component that permits input by auser. Such user input may enable the controller 180 to control operationof the mobile terminal 100. The user input interface 123 may include oneor more of a mechanical input element (for example, a key, a buttonlocated on a front and/or rear surface or a side surface of the mobileterminal 100, a dome switch, a jog wheel, a jog switch, and the like),or a touch-sensitive input, among others. As one example, thetouch-sensitive input may be a virtual key or a soft key, which isdisplayed on a touch screen through software processing, or a touch keywhich is located on the mobile terminal at a location that is other thanthe touch screen. On the other hand, the virtual key or the visual keymay be displayed on the touch screen in various shapes, for example,graphic, text, icon, video, or a combination thereof.

The sensing unit 140 is generally configured to sense one or more ofinternal information of the mobile terminal, surrounding environmentinformation of the mobile terminal, user information, or the like. Thecontroller 180 generally cooperates with the sending unit 140 to controloperation of the mobile terminal 100 or execute data processing, afunction or an operation associated with an application programinstalled in the mobile terminal based on the sensing provided by thesensing unit 140. The sensing unit 140 may be implemented using any of avariety of sensors, some of which will now be described in more detail.

The proximity sensor 141 may include a sensor to sense presence orabsence of an object approaching a surface, or an object located near asurface, by using an electromagnetic field, infrared rays, or the likewithout a mechanical contact. The proximity sensor 141 may be arrangedat an inner region of the mobile terminal covered by the touch screen,or near the touch screen.

The proximity sensor 141, for example, may include any of a transmissivetype photoelectric sensor, a direct reflective type photoelectricsensor, a mirror reflective type photoelectric sensor, a high-frequencyoscillation proximity sensor, a capacitance type proximity sensor, amagnetic type proximity sensor, an infrared rays proximity sensor, andthe like. When the touch screen is implemented as a capacitance type,the proximity sensor 141 can sense proximity of a pointer relative tothe touch screen by changes of an electromagnetic field, which isresponsive to an approach of an object with conductivity. In this case,the touch screen (touch sensor) may also be categorized as a proximitysensor.

The term “proximity touch” will often be referred to herein to denotethe scenario in which a pointer is positioned to be proximate to thetouch screen without contacting the touch screen. The term “contacttouch” will often be referred to herein to denote the scenario in whicha pointer makes physical contact with the touch screen. For the positioncorresponding to the proximity touch of the pointer relative to thetouch screen, such position will correspond to a position where thepointer is perpendicular to the touch screen. The proximity sensor 141may sense proximity touch, and proximity touch patterns (for example,distance, direction, speed, time, position, moving status, and thelike).

In general, the controller 180 processes data corresponding to proximitytouches and proximity touch patterns sensed by the proximity sensor 141,and cause output of visual information on the touch screen. In addition,the controller 180 can control the mobile terminal 100 to executedifferent operations or process different data according to whether atouch with respect to a point on the touch screen is either a proximitytouch or a contact touch.

A touch sensor can sense a touch applied to the touch screen, such asdisplay 151, using any of a variety of touch methods. Examples of suchtouch methods include a resistive type, a capacitive type, an infraredtype, and a magnetic field type, among others.

As one example, the touch sensor may be configured to convert changes ofpressure applied to a specific part of the display 151, or convertcapacitance occurring at a specific part of the display 151, intoelectric input signals. The touch sensor may also be configured to sensenot only a touched position and a touched area, but also touch pressureand/or touch capacitance. A touch object is generally used to apply atouch input to the touch sensor. Examples of typical touch objectsinclude a finger, a touch pen, a stylus pen, a pointer, or the like.

When a touch input is sensed by a touch sensor, corresponding signalsmay be transmitted to a touch controller. The touch controller mayprocess the received signals, and then transmit corresponding data tothe controller 180. Accordingly, the controller 180 may sense whichregion of the display 151 has been touched. Here, the touch controllermay be a component separate from the controller 180, the controller 180,and combinations thereof.

In some embodiments, the controller 180 may execute the same ordifferent controls according to a type of touch object that touches thetouch screen or a touch key provided in addition to the touch screen.Whether to execute the same or different control according to the objectwhich provides a touch input may be decided based on a current operatingstate of the mobile terminal 100 or a currently executed applicationprogram, for example.

The touch sensor and the proximity sensor may be implementedindividually, or in combination, to sense various types of touches. Suchtouches includes a short (or tap) touch, a long touch, a multi-touch, adrag touch, a flick touch, a pinch-in touch, a pinch-out touch, a swipetouch, a hovering touch, and the like.

If desired, an ultrasonic sensor may be implemented to recognizeposition information relating to a touch object using ultrasonic waves.The controller 180, for example, may calculate a position of a wavegeneration source based on information sensed by an illumination sensorand a plurality of ultrasonic sensors. Since light is much faster thanultrasonic waves, the time for which the light reaches the opticalsensor is much shorter than the time for which the ultrasonic wavereaches the ultrasonic sensor. The position of the wave generationsource may be calculated using this fact. For instance, the position ofthe wave generation source may be calculated using the time differencefrom the time that the ultrasonic wave reaches the sensor based on thelight as a reference signal.

The camera 121 typically includes at least one a camera sensor (CCD,CMOS etc.), a photo sensor (or image sensors), and a laser sensor.

Implementing the camera 121 with a laser sensor may allow detection of atouch of a physical object with respect to a 3D stereoscopic image. Thephoto sensor may be laminated on, or overlapped with, the displaydevice. The photo sensor may be configured to scan movement of thephysical object in proximity to the touch screen. In more detail, thephoto sensor may include photo diodes and transistors at rows andcolumns to scan content received at the photo sensor using an electricalsignal which changes according to the quantity of applied light. Namely,the photo sensor may calculate the coordinates of the physical objectaccording to variation of light to thus obtain position information ofthe physical object.

The display 151 is generally configured to output information processedin the mobile terminal 100. For example, the display 151 may displayexecution screen information of an application program executing at themobile terminal 100 or user interface (UI) and graphic user interface(GUI) information in response to the execution screen information.

In some embodiments, the display 151 may be implemented as astereoscopic display for displaying stereoscopic images. A typicalstereoscopic display may employ a stereoscopic display scheme such as astereoscopic scheme (a glass scheme), an auto-stereoscopic scheme(glassless scheme), a projection scheme (holographic scheme), or thelike.

In general, a 3D stereoscopic image may include a left image (e.g., aleft eye image) and a right image (e.g., a right eye image). Accordingto how left and right images are combined into a 3D stereoscopic image,a 3D stereoscopic imaging method can be divided into a top-down methodin which left and right images are located up and down in a frame, anL-to-R (left-to-right or side by side) method in which left and rightimages are located left and right in a frame, a checker board method inwhich fragments of left and right images are located in a tile form, aninterlaced method in which left and right images are alternately locatedby columns or rows, and a time sequential (or frame by frame) method inwhich left and right images are alternately displayed on a time basis.

Also, as for a 3D thumbnail image, a left image thumbnail and a rightimage thumbnail can be generated from a left image and a right image ofan original image frame, respectively, and then combined to generate asingle 3D thumbnail image. In general, the term “thumbnail” may be usedto refer to a reduced image or a reduced still image. A generated leftimage thumbnail and right image thumbnail may be displayed with ahorizontal distance difference there between by a depth corresponding tothe disparity between the left image and the right image on the screen,thereby providing a stereoscopic space sense.

A left image and a right image required for implementing a 3Dstereoscopic image may be displayed on the stereoscopic display using astereoscopic processing unit. The stereoscopic processing unit canreceive the 3D image and extract the left image and the right image, orcan receive the 2D image and change it into a left image and a rightimage.

The audio output module 152 is generally configured to output audiodata. Such audio data may be obtained from any of a number of differentsources, such that the audio data may be received from the wirelesscommunication unit 110 or may have been stored in the memory 170. Theaudio data may be output during modes such as a signal reception mode, acall mode, a record mode, a voice recognition mode, a broadcastreception mode, and the like. The audio output module 152 can provideaudible output related to a particular function (e.g., a call signalreception sound, a message reception sound, etc.) performed by themobile terminal 100. The audio output module 152 may also be implementedas a receiver, a speaker, a buzzer, or the like.

A haptic module 153 can be configured to generate various tactileeffects that a user feels, perceive, or otherwise experience. A typicalexample of a tactile effect generated by the haptic module 153 isvibration. The strength, pattern and the like of the vibration generatedby the haptic module 153 can be controlled by user selection or settingby the controller. For example, the haptic module 153 may outputdifferent vibrations in a combining manner or a sequential manner.

Besides vibration, the haptic module 153 can generate various othertactile effects, including an effect by stimulation such as a pinarrangement vertically moving to contact skin, a spray force or suctionforce of air through a jet orifice or a suction opening, a touch to theskin, a contact of an electrode, electrostatic force, an effect byreproducing the sense of cold and warmth using an element that canabsorb or generate heat, and the like.

The haptic module 153 can also be implemented to allow the user to feela tactile effect through a muscle sensation such as the user's fingersor arm, as well as transferring the tactile effect through directcontact. Two or more haptic modules 153 may be provided according to theparticular configuration of the mobile terminal 100.

An optical output module 154 can output a signal for indicating an eventgeneration using light of a light source. Examples of events generatedin the mobile terminal 100 may include message reception, call signalreception, a missed call, an alarm, a schedule notice, an emailreception, information reception through an application, and the like.

A signal output by the optical output module 154 may be implemented insuch a manner that the mobile terminal emits monochromatic light orlight with a plurality of colors. The signal output may be terminated asthe mobile terminal senses that a user has checked the generated event,for example.

The interface 160 serves as an interface for external devices to beconnected with the mobile terminal 100. For example, the interface 160can receive data transmitted from an external device, receive power totransfer to elements and components within the mobile terminal 100, ortransmit internal data of the mobile terminal 100 to such externaldevice. The interface 160 may include wired or wireless headset ports,external power supply ports, wired or wireless data ports, memory cardports, ports for connecting a device having an identification module,audio input/output (I/O) ports, video I/O ports, earphone ports, or thelike.

The identification module may be a chip that stores various informationfor authenticating authority of using the mobile terminal 100 and mayinclude a user identity module (UIM), a subscriber identity module(SIM), a universal subscriber identity module (USIM), and the like. Inaddition, the device having the identification module (also referred toherein as an “identifying device”) may take the form of a smart card.Accordingly, the identifying device can be connected with the terminal100 via the interface 160.

When the mobile terminal 100 is connected with an external cradle, theinterface 160 can serve as a passage to allow power from the cradle tobe supplied to the mobile terminal 100 or may serve as a passage toallow various command signals input by the user from the cradle to betransferred to the mobile terminal there through. Various commandsignals or power input from the cradle may operate as signals forrecognizing that the mobile terminal is properly mounted on the cradle.

The memory 170 can store programs to support operations of thecontroller 180 and store input/output data (for example, phonebook,messages, still images, videos, etc.). The memory 170 may store datarelated to various patterns of vibrations and audio which are output inresponse to touch inputs on the touch screen.

The memory 170 may include one or more types of storage mediumsincluding a Flash memory, a hard disk, a solid state disk, a silicondisk, a multimedia card micro type, a card-type memory (e.g., SD or DXmemory, etc.), a Random Access Memory (RAM), a Static Random AccessMemory (SRAM), a Read-Only Memory (ROM), an Electrically ErasableProgrammable Read-Only Memory (EEPROM), a Programmable Read-Only memory(PROM), a magnetic memory, a magnetic disk, an optical disk, and thelike. The mobile terminal 100 may also be operated in relation to anetwork storage device that performs the storage function of the memory170 over a network, such as the Internet.

The controller 180 may typically control the general operations of themobile terminal 100. For example, the controller 180 may set or releasea lock state for restricting a user from inputting a control commandwith respect to applications when a status of the mobile terminal meetsa preset condition.

The controller 180 can also perform the controlling and processingassociated with voice calls, data communications, video calls, and thelike, or perform pattern recognition processing to recognize ahandwriting input or a picture drawing input performed on the touchscreen as characters or images, respectively. In addition, thecontroller 180 can control one or a combination of those components inorder to implement various exemplary embodiments disclosed herein.

The power supply unit 190 receives external power or provide internalpower and supply the appropriate power required for operating respectiveelements and components included in the mobile terminal 100. The powersupply unit 190 may include a battery, which is typically rechargeableor be detachably coupled to the terminal body for charging.

The power supply unit 190 may include a connection port. The connectionport may be configured as one example of the interface 160 to which anexternal charger for supplying power to recharge the battery iselectrically connected.

As another example, the power supply unit 190 may be configured torecharge the battery in a wireless manner without use of the connectionport. In this example, the power supply unit 190 can receive power,transferred from an external wireless power transmitter, using at leastone of an inductive coupling method which is based on magnetic inductionor a magnetic resonance coupling method which is based onelectromagnetic resonance.

Various embodiments described herein may be implemented in acomputer-readable medium, a machine-readable medium, or similar mediumusing, for example, software, hardware, or any combination thereof.

Referring now to FIGS. 1B and 1C, the mobile terminal 100 is describedwith reference to a bar-type terminal body. However, the mobile terminal100 may alternatively be implemented in any of a variety of differentconfigurations. Examples of such configurations include watch-type,clip-type, glasses-type, or as a folder-type, flip-type, slide-type,swing-type, and swivel-type in which two and more bodies are combinedwith each other in a relatively movable manner, and combinationsthereof. Discussion herein will often relate to a particular type ofmobile terminal (for example, bar-type, watch-type, glasses-type, andthe like). However, such teachings with regard to a particular type ofmobile terminal will generally apply to other types of mobile terminalsas well.

Here, the terminal body may be understood as referring to the mobileterminal 100 as at least one assembly.

The mobile terminal 100 will generally include a case (for example,frame, housing, cover, and the like) forming the appearance of theterminal. In this embodiment, the case is formed using a front case 101and a rear case 102. Various electronic components are incorporated intoa space formed between the front case 101 and the rear case 102. Atleast one middle case may be additionally positioned between the frontcase 101 and the rear case 102.

The display 151 is shown located on the front side of the terminal bodyto output information. As illustrated, a window 151 a of the display 151may be mounted to the front case 101 to form the front surface of theterminal body together with the front case 101.

In some embodiments, electronic components may also be mounted to therear case 102. Examples of such electronic components include adetachable battery 191, an identification module, a memory card, and thelike. Rear cover 103 is shown covering the electronic components, andthis cover may be detachably coupled to the rear case 102. Therefore,when the rear cover 103 is detached from the rear case 102, theelectronic components mounted to the rear case 102 are externallyexposed.

As illustrated, when the rear cover 103 is coupled to the rear case 102,a side surface of the rear case 102 is partially exposed. In some cases,upon the coupling, the rear case 102 may also be completely shielded bythe rear cover 103. In some embodiments, the rear cover 103 may includean opening for externally exposing a camera 121 b or an audio outputmodule 152 b.

The cases 101, 102, 103 may be formed by injection-molding syntheticresin or may be formed of a metal, for example, stainless steel (STS),aluminum (Al), titanium (Ti), or the like.

As an alternative to the example in which the plurality of cases form aninner space for accommodating components, the mobile terminal 100 may beconfigured such that one case forms the inner space. In this example, amobile terminal 100 having a uni-body is formed in such a manner thatsynthetic resin or metal extends from a side surface to a rear surface.

If desired, the mobile terminal 100 may include a waterproofing unit(not shown) for preventing introduction of water into the terminal body.For example, the waterproofing unit may include a waterproofing memberwhich is located between the window 151 a and the front case 101,between the front case 101 and the rear case 102, or between the rearcase 102 and the rear cover 103, to hermetically seal an inner spacewhen those cases are coupled.

The mobile terminal includes a display 151, a first and a second audiooutput modules 151 a/151 b, a proximity sensor 141, an illuminationsensor 142, an optical output module 154, a first and a second cameras121 a/121 b, a first and a second manipulation units 123 a/123 b, amicrophone 122, interface 160 and the like.

It will be described for the mobile terminal as shown in FIGS. 1B and1C. The display 151, the first audio output module 151 a, the proximitysensor 141, an illumination sensor 142, the optical output module 154,the first camera 121 a and the first manipulation unit 123 a arearranged in front surface of the terminal body, the second manipulationunit 123 b, the microphone 122 and interface 160 are arranged in sidesurface of the terminal body, and the second audio output modules 151 band the second camera 121 b are arranged in rear surface of the terminalbody.

However, it is to be understood that alternative arrangements arepossible and within the teachings of the instant disclosure. Somecomponents may be omitted or rearranged. For example, the firstmanipulation unit 123 a may be located on another surface of theterminal body, and the second audio output module 152 b may be locatedon the side surface of the terminal body.

The display 151 displays (outputs) information processed in the mobileterminal 100. The display 151 may execution screen information of anapplication program executed on the mobile terminal 100 or userinterface (UI) or graphical user interface (GUI) information accordingto such execution screen information.

Examples of such suitable display devices include a liquid crystaldisplay (LCD), a thin film transistor-liquid crystal display (TFT-LCD),an organic light emitting diode (OLED), a flexible display, a3-dimensional (3D) display, an e-ink display, and combinations thereof.

The display 151 may be implemented using two display devices, which canimplement the same or different display technology. For instance, aplurality of the displays 151 may be arranged on one side, either spacedapart from each other, or these devices may be integrated, or thesedevices may be arranged on different surfaces.

The display 151 may also include a touch sensor which senses a touchinput received at the display. When a touch is input to the display 151,the touch sensor may be configured to sense this touch and thecontroller 180, for example, may generate a control command or othersignal corresponding to the touch. The content which is input in thetouching manner may be a text or numerical value, or a menu item whichcan be indicated or designated in various modes.

The touch sensor may be configured in a form of a film having a touchpattern, disposed between the window 151 a and a display on a rearsurface of the window 151 a, or a metal wire which is patterned directlyon the rear surface of the window 151 a. Alternatively, the touch sensormay be integrally formed with the display. For example, the touch sensormay be disposed on a substrate of the display or within the display.

The display 151 may also form a touch screen together with the touchsensor. Here, the touch screen may serve as the user input interface 123(see FIG. 1A). Therefore, the touch screen may replace at least some ofthe functions of the first manipulation unit 123 a.

The first audio output module 152 a may be implemented in the form of aspeaker to output voice audio, alarm sounds, multimedia audioreproduction, and the like.

The window 151 a of the display 151 will typically include an apertureto permit audio generated by the first audio output module 152 a topass. One alternative is to allow audio to be released along an assemblygap between the structural bodies (for example, a gap between the window151 a and the front case 101). In this case, a hole independently formedto output audio sounds may not be seen or is otherwise hidden in termsof appearance, thereby further simplifying the appearance andmanufacturing of the mobile terminal 100.

The optical output module 154 can be configured to output light forindicating an event generation. Examples of such events include amessage reception, a call signal reception, a missed call, an alarm, aschedule notice, an email reception, information reception through anapplication, and the like. When a user has checked a generated event,the controller can control the optical output interface 154 to stop thelight output.

The first camera 121 a can process image frames such as still or movingimages obtained by the image sensor in a capture mode or a video callmode. The processed image frames can then be displayed on the display151 or stored in the memory 170.

The first and second manipulation units 123 a and 123 b are examples ofthe user input interface 123, which may be manipulated by a user toprovide input to the mobile terminal 100. The first and secondmanipulation units 123 a and 123 b may also be commonly referred to as amanipulating portion, and may employ any tactile method that allows theuser to perform manipulation such as touch, push, scroll, or the like.The first and second manipulation units 123 a and 123 b may also employany non-tactile method that allows the user to perform manipulation suchas proximity touch, hovering, or the like.

FIG. 1B illustrates the first manipulation unit 123 a as a touch key,but possible alternatives include a mechanical key, a push key, a touchkey, and combinations thereof.

Input received at the first and second manipulation units 123 a and 123b may be used in various ways. For example, the first manipulation unit123 a may be used by the user to provide an input to a menu, home key,cancel, search, or the like, and the second manipulation unit 123 b maybe used by the user to provide an input to control a volume level beingoutput from the first or second audio output modules 152 a or 152 b, toswitch to a touch recognition mode of the display 151, or the like.

As another example of the user input interface 123, a rear inputinterface (not shown) may be located on the rear surface of the terminalbody. The rear input interface can be manipulated by a user to provideinput to the mobile terminal 100. The input may be used in a variety ofdifferent ways. For example, the rear input interface may be used by theuser to provide an input for power on/off, start, end, scroll, controlvolume level being output from the first or second audio output modules152 a or 152 b, switch to a touch recognition mode of the display 151,and the like. The rear input interface may be configured to permit touchinput, a push input, or combinations thereof.

The rear input interface may be located to overlap the display 151 ofthe front side in a thickness direction of the terminal body. As oneexample, the rear input interface may be located on an upper end portionof the rear side of the terminal body such that a user can easilymanipulate it using a forefinger when the user grabs the terminal bodywith one hand. Alternatively, the rear input interface can be positionedat most any location of the rear side of the terminal body.

Embodiments that include the rear input interface may implement some orall of the functionality of the first manipulation unit 123 a in therear input interface. As such, in situations where the firstmanipulation unit 123 a is omitted from the front side, the display 151can have a larger screen.

As a further alternative, the mobile terminal 100 may include a fingerscan sensor which scans a user's fingerprint. The controller 180 canthen use fingerprint information sensed by the finger scan sensor aspart of an authentication procedure. The finger scan sensor may also beinstalled in the display 151 or implemented in the user input interface123.

The microphone 122 is shown located at an end of the mobile terminal100, but other locations are possible. If desired, multiple microphonesmay be implemented, with such an arrangement permitting the receiving ofstereo sounds.

The interface 160 may serve as a path allowing the mobile terminal 100to interface with external devices. For example, the interface 160 mayinclude one or more of a connection terminal for connecting to anotherdevice (for example, an earphone, an external speaker, or the like), aport for near field communication (for example, an Infrared DataAssociation (IrDA) port, a Bluetooth port, a wireless LAN port, and thelike), or a power supply terminal for supplying power to the mobileterminal 100. The interface 160 may be implemented in the form of asocket for accommodating an external card, such as SubscriberIdentification Module (SIM), User Identity Module (UIM), or a memorycard for information storage.

The second camera 121 b is shown located at the rear side of theterminal body and includes an image capturing direction that issubstantially opposite to the image capturing direction of the firstcamera unit 121 a. If desired, second camera 121 a may alternatively belocated at other locations, or made to be moveable, in order to have adifferent image capturing direction from that which is shown.

The second camera 121 b can include a plurality of lenses arranged alongat least one line. The plurality of lenses may also be arranged in amatrix configuration. The cameras may be referred to as an “arraycamera.” When the second camera 121 b is implemented as an array camera,images may be captured in various manners using the plurality of lensesand images with better qualities.

As shown in FIG. 1C, a flash 124 is shown adjacent to the second camera121 b. When an image of a subject is captured with the camera 121 b, theflash 124 may illuminate the subject.

As shown in FIG. 1B, the second audio output module 152 b can be locatedon the terminal body. The second audio output module 152 b may implementstereophonic sound functions in conjunction with the first audio outputmodule 152 a, and may be also used for implementing a speaker phone modefor call communication.

At least one antenna for wireless communication may be located on theterminal body. The antenna may be installed in the terminal body orformed by the case. For example, an antenna which configures a part ofthe broadcast receiving module 111 may be retractable into the terminalbody. Alternatively, an antenna may be formed using a film attached toan inner surface of the rear cover 103, or a case that includes aconductive material.

A power supply unit 190 for supplying power to the mobile terminal 100may include a battery 191, which is mounted in the terminal body ordetachably coupled to an outside of the terminal body.

The battery 191 may receive power via a power source cable connected tothe interface 160. Also, the battery 191 can be recharged in a wirelessmanner using a wireless charger. Wireless charging may be implemented bymagnetic induction or electromagnetic resonance.

The rear cover 103 is shown coupled to the rear case 102 for shieldingthe battery 191, to prevent separation of the battery 191, and toprotect the battery 191 from an external impact or from foreignmaterial. When the battery 191 is detachable from the terminal body, therear case 103 may be detachably coupled to the rear case 102.

An accessory for protecting an appearance or assisting or extending thefunctions of the mobile terminal 100 can also be provided on the mobileterminal 100. As one example of an accessory, a cover or pouch forcovering or accommodating at least one surface of the mobile terminal100 may be provided. The cover or pouch may cooperate with the display151 to extend the function of the mobile terminal 100. Another exampleof the accessory is a touch pen for assisting or extending a touch inputto a touch screen.

In the present disclosure, information processed in the mobile terminalmay be displayed using a flexible display. Hereinafter, this will bedescribed in greater detail with reference to the accompanying drawings.

Hereinafter, embodiments related to a control method which may beimplemented in the above-described mobile terminal will be describedwith reference to the accompanying drawings. It is apparent to thoseskilled in the art that the present disclosure can be embodied in otherspecific forms without departing from the spirit and essential featuresof the present disclosure.

In this specification, the term memory 170 may be used interchangeablywith the term storage unit 170.

The mobile terminal 100 may include a communication unit composed of thewireless communication unit 110 and the interface 160.

The controller 180 may control operation of each component of the mobileterminal 100 under control of the artificial intelligence part 130.

The input unit 120 of the mobile terminal 100 may include the sensingunit 140, and perform all functions performed by the sensing unit 140.For example, the input unit 120 may detect user touch input.

Hereinafter, a mobile terminal will be described as an embodiment of thepresent disclosure. However, the present disclosure is not limitedthereto and is applicable to all terminals or electronic apparatuseshaving an artificial intelligence function.

That is, the mobile terminal described with reference to FIG. 1 ismerely an example of a terminal capable of implementing the presentdisclosure. Accordingly, the terminal, to which the present disclosureis applied, may not be a mobile terminal. In addition, the terminal, towhich the present disclosure is applied, may include some or all of thecomponents described with reference to FIG. 1.

FIG. 2 is a view illustrating a method of operating a terminal accordingto an embodiment of the present disclosure.

In FIG. 2, the method of operating the terminal according to theembodiment of the present disclosure may include step S210 of acquiringinformation on the situation of a user and step of outputting anotification when the situation of the user corresponds to informationincluded in a database.

The terminal 310 may include at least one of an output unit, a storageunit, a situation information acquisition unit, a communication unit, acontroller or an artificial intelligence part.

For the output unit of the terminal 310, refer to the description of theoutput unit 150 of the mobile terminal 100 described with reference toFIG. 1.

The output unit may output a notification. The notification may be toprovide a guide to a user's behavior and may be output using variousmethods.

For example, the output unit may visually display the notification,audibly output the notification, output the notification using a hapticsignal or output the notification using a combination of theabove-described methods.

For the storage unit of the terminal 310, refer to the description ofthe storage unit 180 of the mobile terminal 100 described with referenceto FIG. 1.

The storage unit may store various types of data or programs necessaryto drive the terminal.

In addition, the storage unit may store a database. Here, the databasemay include at least one of a user's personal database, a standardbehavior database or an accident type database.

For the situation information acquisition unit of the terminal 310,refer to the description of the wireless communication unit 110, theinput unit 120 and the sensing unit 140 of the mobile terminal 100described with reference to FIG. 1.

The situation information acquisition unit may collect data necessary todetermine the situation of the user. In this case, the artificialintelligence part may acquire information on the situation of the userbased on the data acquired by the situation information acquisitionunit.

For the communication unit, refer to the description of the wirelesscommunication unit 110 and the interface 160 of the mobile terminal 100described with reference to FIG. 1.

The communication unit may communicate with one or more externalapparatus. For example, if the terminal configures a driver guidesystem, the communication unit may communicate with at least one of adriver recognition device, a behavior detection device, a driving statedetection device or a display device.

For the controller, refer to the description of the controller 180 ofthe mobile terminal 100 described with reference to FIG. 1.

The controller may control overall operation of the terminal.Specifically, the controller may control operation of at least one ofthe output unit, the storage unit, the situation information acquisitionunit, the communication unit or the artificial intelligence part.

For the artificial intelligence part, refer to the description of theartificial intelligence part 130 of the mobile terminal 100 describedwith reference to FIG. 1.

The artificial intelligence part may output a notification when thesituation of the user corresponds to information included in thedatabase.

The notification may be output via the output unit or an output device350. Specifically, the artificial intelligence part may control theoutput unit of the terminal 310 to output the notification. In addition,the artificial intelligence part may control the output device 350 tooutput the notification, by transmitting a notification output signal tothe output device 350.

The artificial intelligence part may directly control the output unit tooutput the notification or the artificial intelligence part may controla controller for controlling output of the notification to output thenotification.

FIG. 3 is a view illustrating a driver guide system including a terminalaccording to an embodiment of the present disclosure.

In FIG. 3, the driver guide system 300 may include a terminal 310. Inaddition, the driver guide system 300 may include at least one of adriver recognition device 320, a behavior detection device 300, adriving state detection device 340 or a display device 350.

The driver recognition device 320, the behavior detection device 300,the driving state detection device 340 and the display device 350 may bemounted in a vehicle.

The terminal 310 configuring the driver guide system 300 may be themobile terminal 100 described with reference to FIG. 1, and, if a usercarrying the mobile terminal 100 rides in the vehicle, the mobileterminal 100 may perform wireless communication with the driverrecognition device 320, the behavior detection device 330, the drivingstate detection device 340 and the display device 350.

For the driver recognition device 320, the behavior detection device330, the driving state detection device 340 and the display device 350,refer to some or all of the components of the mobile terminal 100described with reference to FIG. 1.

The driver recognition device 320 may acquire data for determining whothe driver is. Specifically, the driver recognition device 320 maycollect unique information of a key of each driver, the fingerprint,grip, size and temperature of the driver's hand when each driver grabs ahandle in order to open the door of the vehicle, iris recognitioninformation, retina recognition information, speech recognitioninformation, etc. In addition, the driver recognition device 320 maycapture the image of the driver to acquire facial recognitioninformation.

To this end, the driver recognition device 320 may include a camera, amicrophone, a fingerprint scanner, an iris scanner, a retina scanner, asensing device such as a temperature sensor, etc.

In this case, the driver recognition device 320 may transmit theacquired data to the terminal 310. In addition, the artificialintelligence part of the terminal 310 may determine who is driving basedon the received data.

The behavior detection device 330 may acquire data for determining thebehavior of the user. Specifically, the behavior detection device 330may include one or more cameras and one or more cameras may capture theimage of the user to acquire behavior information.

In addition, the behavior detection device 300 may transmit the acquireddata to the terminal 310. In this case, the artificial intelligence partof the terminal 310 may determine what the behavior of the user is basedon the received data.

The driving state detection device 340 may acquire data for determiningthe driving state of the vehicle.

Specifically, the driving state detection device 340 may capture theimage of the surroundings of the vehicle.

In addition, the driving state detection device 340 may include a GPSmodule for acquiring the position of the vehicle.

In addition, the driving state detection device 340 may sense a signalrelated to driving of the vehicle 100. To this end, the sensing unit 130may include a collision sensor, a wheel sensor, a speed sensor, aninclination sensor, a weight sensor, a heading sensor, a yaw sensor, agyro sensor, a position module, a vehicle forward/reverse sensor, abattery sensor, a fuel sensor, a tire sensor, a steering sensoraccording to a steering wheel rotation, a vehicle inside-temperaturesensor, a vehicle inside-humidity sensor, an ultrasonic sensor, a radar,a lidar, etc.

Accordingly, based on the data acquired by the driving state detectiondevice 340, the artificial intelligence part of the terminal 310 mayacquire vehicle collision information, vehicle direction information,vehicle location information, vehicle angle information, vehicle speedinformation, vehicle acceleration information, vehicle slopeinformation, vehicle forward/reverse information, battery information,fuel information, tire information, vehicle lamp information, vehicleinside-temperature information, vehicle inside-humidity information,lane change information, etc.

The output device 350 outputs information processed in the terminal 310and may include at least one of a display, a sound output unit or ahaptic output unit.

The display of the output device 350 may display information processedin the terminal 310. For example, the display of the output device 350may display the notification output from the terminal 310.

The display of the output device 350 may include at least one of aliquid crystal display (LCD), a thin film transistor-liquid crystaldisplay (TFT LCD), an organic light-emitting diode (OLED), a flexibledisplay, a three-dimensional 3D) display, or an e-ink display.

The display of the output device 350 may have an inter-layered structureor an integrated structure with a touch sensor so as to implement atouchscreen. The touchscreen may provide an output interface between thevehicle and a user, as well as functioning as the input unit whichprovides an input interface between the vehicle and the user. In thiscase, the display of the output device 350 may include a touch sensorfor detecting touch of the display 141 to receive a control commandusing a touch method. Using this, when the display of the output device350 is touched, the touch sensor detects the touch and the terminal 310may generate a control command corresponding to the touch. Content inputusing the touch method may be characters or numbers or a menu itemindicated or designated in various modes.

The display of the output device 350 may be implemented as a head updisplay (HUD). If the display of the output device 350 is implemented asa HUD, information may be output via a transparent display provided in awindshield. Alternatively, the display of the output device 350 mayinclude a projection module to output information via an image projectedonto the windshield.

The sound output unit of the output device 350 may convert thenotification into sound and output the sound. Specifically, the soundoutput unit of the output device 350 may receive an electrical signalcorresponding to the notification from the terminal 310, convert thereceived electrical signal into an audio signal, and output the audiosignal. To this end, the sound output unit of the output device 350 mayinclude a speaker, etc.

The haptic output unit of the output device 350 may generate tactileoutput corresponding to the notification. For example, the haptic outputunit of the output device 350 may vibrate a steering wheel, a seat beltand a seat such that the user recognizes the output.

The terminal 310 may include some or all of the components of the driverrecognition device 320, the behavior detection device 330 and thedriving state detection device 340 described with reference to FIG. 3.In addition, the terminal 310 may perform the functions of the driverrecognition device 320, the behavior detection device 330 and thedriving state detection device 340 described with respect to FIG. 3.

For example, the terminal 310 may acquire data for determining thebehavior of the user or data for determining who the driver is, bycapturing the image of the driver.

The terminal 310 may communicate with the devices 320, 330, 340 and 350configuring the driver guide system 300 via the communication unit ofthe terminal 310 and control overall operation of the devices 320, 330,340 and 350 configuring the driver guide system 300.

In addition, the artificial intelligence part of the terminal 310 mayreceive data from the devices 320, 330, 340 and 350 configuring thedriver guide system 300 and acquire information on the situation of theuser based on the received data.

In addition, the artificial intelligence part of the terminal 310 maytransmit a signal corresponding to the notification to the output device350 based on the situation of the user. The output device 350 may outputthe notification based on the received signal.

In this specification, “the terminal 310 outputting the notification” or“the artificial intelligence part outputting the notification” may meanthat the artificial intelligence part controls the output unit of theterminal 310 such that the notification is output from the terminal 310or the artificial intelligence part transmits the signal correspondingto the notification to the output device 350 such that the notificationis output from the terminal 310.

In this specification, “the artificial intelligence part acquiring theinformation on the situation of the user” may mean that the informationon the situation of the user is acquired based on at least one of thedata received from the other devices 320, 330, 340 and 350 configuringthe system 300 or the data acquired by the situation informationacquisition unit of the terminal 310.

FIGS. 4 to 5 are views illustrating a method of outputting anotification according to a user's behavior according to an embodimentof the present disclosure.

The method of operating the terminal 310 according to the embodiment ofthe present disclosure may include step S410 of acquiring information onthe behavior of a user and step S420 of outputting a notification whenthe behavior of the user corresponds to the behavior of a driverincluded in a standard behavior database.

Step S410 of acquiring the information on the behavior of the user willnow be described.

The artificial intelligence part of the terminal 310 may acquireinformation on the situation of the user. Specifically, the artificialintelligence part of the terminal 310 may acquire the information on thebehavior of the user.

Specifically, based on at least one of the data received from the otherdevices 320, 330, 340 and 350 configuring the system 300 or the dataacquired by the situation information acquisition unit of the terminal310, the artificial intelligence part may acquire the information on thebehavior of the user.

For example, when the driving state detection device detects lane changeof a vehicle and turn-on of a turn signal, the artificial intelligencepart may determine that operation of turning on the turn signal is latewhen the user changes a lane based on a lane change time and a turn-ontime of the turn signal.

In another example, when the driving state detection device detects lanechange of a vehicle and the situation information acquisition unitphotographs or films user's operation of turning on the turn signal, theartificial intelligence part may determine that user's operation ofturning on the turn signal is late when the user changes a lane, basedon a lane change time and a time when operation of turning on the turnsignal is photographed or filmed.

In another example, when the behavior detection device 330 captures theimage of the user, the artificial intelligence part may determine thatthe user does not view a rearview mirror for a predetermined time ormore based on the captured image.

In FIG. 5, the behavior detection device 330 for capturing the image ofthe user is shown.

A database may be stored in the storage unit of the terminal 310.

Here, the database may include a standard behavior database.

The standard behavior database may include information on the behaviorof the driver while driving. Here, the information on the behavior ofthe driver while driving may be information obtained by collecting thebehaviors of the drivers when a plurality of accidents occurs.

Specifically, a standard behavior database provider may acquire theimage of the driver captured by the behavior detection device 330 whenan accident occurs. When such images are acquired from the plurality ofaccident cases, the standard behavior database provider may analyze thebehavior pattern of the driver when an accident occurs and generate thestandard behavior database including information on the behavior of thedriver while driving. Meanwhile, the controller may receive the standardbehavior database from the standard behavior database provider and storethe standard behavior database in the storage unit.

For example, from a plurality of accident cases, it may be derived thata behavior of watching a smartphone while driving causes an accident. Inthis case, information on the behavior of the driver while driving maybe information on the behavior of watching the smartphone while driving.

In another example, from a plurality of accident cases, it may bederived that a behavior of viewing a passenger seat without keeping eyesforward causes an accident. In this case, information on the behavior ofthe driver while driving may be information on the behavior of viewingthe passenger seat without keeping eyes forward.

Although the behavior of the driver while driving is described as beingacquired from the plurality of accident cases, the behavior of thedriver while driving is not limited thereto and may be a behavior set bythe standard behavior database provider. For example, the standardbehavior database provider may set a behavior of driving while drowsy asthe information on the behavior of the driver while driving.

In addition, the behavior of the driver while driving may be set byreferring to a law. For example, if the behavior of watching thesmartphone while driving violates the traffic regulations, the behaviorof watching the smartphone may be set as the information on the behaviorof the driver while driving.

Although the behavior of the driver while driving is described as beingacquired from the captured image of the driver, the behavior of thedriver while driving is not limited thereto and may be acquired from thestatement of the accident party or the result of investigation of thepolice.

Step S420 of outputting the notification when the behavior of the usercorresponds to the behavior of the driver included in the standardbehavior database will now be described.

The artificial intelligence part may output the notification when thesituation of the user corresponds to the information included in thedatabase. Specifically, the artificial intelligence part may output thenotification when the behavior of the user corresponds to the behaviorof the driver included in the standard behavior database.

For example, when the artificial intelligence part determines that theuser is currently drowsy based on the behavior of the user and thestandard behavior database includes information on the drowsy behaviorof the driver, the artificial intelligence part may output a warningnotification.

In another example, when the artificial intelligence part determinesthat operation of turning on the turn signal when the user changes thelane is late and the standard behavior database includes the informationon the behavior of turning on the turn signal late, the artificialintelligence part may output the warning notification.

The user may unconsciously or consciously behave dangerously whiledriving. When such a behavior occurs, the artificial intelligence partmay warn the user, thereby providing a guide to user's safe driving. Inparticular, by comparing the standard behavior database acquired fromthe plurality of accident cases with the current behavior of the user, amore realistic guide is possible and the user may be warned.

Meanwhile, although the notification is described as being output basedon the behavior of the user in the above-described embodiment, thenotification may be output in consideration of the position of the userin addition to the behavior of the user.

Specifically, the standard behavior database may include information ona position and information on the behavior of the driver at theposition. Here, the information on the behavior of the driver whiledriving may be obtained by collecting the behaviors of the drivers whena plurality of accidents occurs at the position.

Specifically, in the standard behavior database provider, when accidentsoccurs at various positions, the behavior detection device 330 mayacquire the captured images of the drivers. When such images areacquired from the plurality of accident cases, the standard behaviordatabase provider may analyze the behavior pattern of the driver whenaccidents occur at various positions and generate the standard behaviordatabase including information on the behaviors of the drivers at theplurality of positions. Meanwhile, the controller may receive thestandard behavior database from the standard behavior database providerand store the standard behavior database in the storage unit.

For example, from the plurality of accident cases, it may be derivedthat the pattern of the driver's behavior of looking to the right at afirst position may cause an accident (for example, when the driverpasses the first position, since there is a beautiful building in aright direction, the driver may frequently look to the right and thus anaccident may frequently occur. In this case, information included in thestandard behavior database may be the position information of the firstposition and information on the driver's behavior of looking to theright.

In another example, from the plurality of accident cases, it may bederived that the driver does not turn on the turn signal at a secondposition when changing the lane to the left, thereby causing anaccident. In this case, information included in the standard behaviordatabase may be the position information of the second position andinformation on the driver's behavior of not turning on the turn signalat the second position.

The position included in the database may be a position where a vehicleaccident frequently occurs.

Meanwhile, operation performed by the standard behavior databaseprovider may be performed by the artificial intelligence part of theterminal 310.

The artificial intelligence part of the terminal 310 may receiveinformation on the position of the user via the communication unit ofthe terminal 310 or from the driving state detection device 340 andacquire the information on the position of the user.

In addition, the artificial intelligence part of the terminal 310 mayoutput the notification when the user passes the position included inthe standard behavior database and the behavior of the user correspondsto the behavior of the driver at the position.

For example, when the artificial intelligence part determines that thecurrent position of the user is a first position and the user is drowsybased on the behavior of the user and the standard behavior databaseincludes the first position and information on the driver's drowsybehavior at the first position, the artificial intelligence part mayoutput the warning notification.

In the above-described embodiment, the notification is described asbeing output when the user passes the position. Passing through theposition may include not only passing the position but also being goingto pass the position.

For example, when the first position is a specific intersection and theuser is driving toward the specific intersection, the artificialintelligence part may output the notification at a predetermined timebefore entering the specific intersection (or at a predetermineddistance before the specific intersection).

The patterns of accidents occurring at specific positions may not bestandardized. For example, when a sea suddenly appears on the left side,drivers may look to the left, thereby causing a lot of accidents. Inanother example, drivers who are driving on a straight road are in adrowsy state, thereby causing a lot of accidents.

In the present disclosure, by comparing behaviors causing accidents atspecific positions with the current behavior of the user and providing aguide to the behavior of the user, it is possible to prevent an accidentby a standardized pattern at the specific position.

The standard behavior database may include information on an incorrectbehavior of a driver while driving. For example, the standard behaviordatabase may include information on the behavior of the driver acquiredfrom the plurality of accident cases, information on the behavior of thedriver set by the standard behavior database provider, etc.

The storage unit may include information on a correct behaviorcorresponding to the incorrect behavior of the driver.

For example, the standard behavior database may include a driver'sbehavior of driving while drowsy, and information on a behavior such as“Do not drowse”, “Why don't you open a window and stretch in order tokeep you awake?” or “Stop by a rest area for a cup of coffee” may bestored in the storage unit.

The artificial intelligence part may output a correct behavior when thebehavior of the user corresponds to the incorrect behavior of thedriver.

For example, when the artificial intelligence part determines that theuser is currently drowsy based on the behavior of the user and thestandard behavior database includes information on the drowsy behaviorof the driver, the artificial intelligence part may output a correctbehavior “Stop by a rest area for a cup of coffee”.

FIG. 6 is a view illustrating a method of determining whether anotification is output and the intensity of the notification inconsideration of the behavior pattern of a user according to anembodiment of the present disclosure.

The method of operating the terminal 310 according to the embodiment ofthe present disclosure may include step S610 of acquiring and storingthe behavior pattern of a user in a personal database, step S620 ofacquiring information on the behavior of the user, step S630 ofdetermining whether the behavior of the user corresponds to the behaviorof the driver of the standard behavior database, step S640 ofdetermining whether the behavior of the user corresponds to the behaviorpattern of the user when the behavior of the user corresponds to thebehavior of the driver of the standard behavior database and step S650of determining whether a notification is output or adjusting theintensity of the notification based on whether the behavior of the usercorresponds to the behavior pattern of the user.

The database may include the personal database of the user.

Step S610 of acquiring and storing the behavior pattern of the user inthe personal database will now be described.

Here, information on the situation of the user may be stored in thepersonal database of the user. Specifically, the artificial intelligencepart may store information on the behavior of the user in the personaldatabase of the user.

For example, the artificial intelligence part may store a user'sbehavior of not seeing a rearview mirror for a predetermined time ormore, a behavior of turning on a turn signal late when changing a laneto the left, a user's behavior of looking to the right without keepingtheir eyes forward and a user's drowsy behavior in the personaldatabase.

Meanwhile, the personal database of the user may include information onthe behavior pattern of the user. The behavior pattern of the user maybe acquired based on a plurality of past behaviors of the user.

Specifically, the artificial intelligence part may acquire the behaviorpattern of the user, based on at least one of how often a specificbehavior of the user appears, the number of times that the specificbehavior of the user appears or a point in time when the specificpattern appears.

For example, when the number of times that the user's drowsy behaviorappears is equal to or greater than a predetermined number or is one ormore per month, the artificial intelligence part may determine theuser's drowsy behavior as the behavior pattern of the user and store thedetermined behavior pattern in the personal database.

In another example, although the behavior pattern of the user includes abehavior of turning on the turn signal late one year ago, if thatbehavior have not appeared thereafter, the artificial intelligence partmay exclude the behavior of turning on the turn signal late from thebehavior pattern.

For step S620 of acquiring the information on the behavior of the userand step S630 of determining whether the behavior of the usercorresponds to the behavior of the driver of the standard behaviordatabase, refer to the description of FIGS. 4 to 5.

Step S640 of determining whether the behavior of the user corresponds tothe behavior pattern of the user when the behavior of the usercorresponds to the behavior of the driver of the standard behaviordatabase will now be described.

When the behavior of the user corresponds to the behavior of the driverof the standard behavior database, the artificial intelligence part maydetermine whether the behavior of the user corresponds to the behaviorpattern of the user.

For example, when the behavior of the user is currently a drowsybehavior and the behavior of the driver of the standard behaviordatabase includes a drowsy behavior, the artificial intelligence partmay determine whether the user's drowsy behavior is stored in thepersonal database as the behavior pattern of the user. When the user'sdrowsy behavior is stored in the personal database as the behaviorpattern of the user, the artificial intelligence part may determine thatthe current behavior of the user corresponds to the behavior pattern ofthe user.

Step S650 of determining whether the notification is output or adjustingthe intensity of the notification based on whether the behavior of theuser corresponds to the behavior pattern of the user will now bedescribed.

Based on whether the behavior of the user corresponds to the behaviorpattern of the user, the artificial intelligence part may determinewhether the notification is output.

For example, if the current behavior of the user is a behavior of notkeeping eyes forward and the standard behavior database includes abehavior of not keeping eyes forward but the behavior of not keepingeyes forward is not the behavior pattern of the user, the artificialintelligence part may not output the notification.

Based on whether the behavior of the user corresponds to the behaviorpattern of the user, the artificial intelligence part may adjust theoutput intensity of the notification. Specifically, the artificialintelligence part may output the notification with a low intensity or ahigh intensity based on whether the behavior of the user corresponds tothe behavior pattern of the user.

For example, if the current behavior of the user is a behavior of notkeeping eyes forward and the standard behavior database includes abehavior of not keeping eyes forward but the behavior of not keepingeyes forward is not the behavior pattern of the user, the artificialintelligence part may output the notification with a low intensity.

In another example, if the current behavior of the user is a behavior ofnot keeping eyes forward, the standard behavior database includes abehavior of not keeping eyes forward, and the behavior of not keepingeyes forward is the behavior pattern of the user, the artificialintelligence part may output the notification with a high intensity.

The notification with the low intensity and the notification with thehigh intensity may be distinguished according to the level, frequency,etc.

For example, when the notification is audibly output, the notificationwith the low intensity may be output once every 5 seconds in a smallvoice and the notification with the high intensity may be output onceevery 2 seconds in a large voice.

In another example, if the notification is output as an image, thenotification with the low intensity may be output in text having a smallsize and the notification with the high intensity may be output in texthaving a large size.

In another example, when the notification is output as a haptic signal,the notification with the low intensity may be output once every 5seconds with a small vibration magnitude and the notification with thehigh intensity may be output once every 2 seconds with a large vibrationmagnitude.

Outputting the notification whenever the specific behavior of the userappears may cause fatigue to the user. Accordingly, the presentdisclosure has an advantage of outputting a warning only with respect tothe bad habits of the user, by extracting the behavior pattern of theuser from the behaviors of the user, determining whether the specificbehavior of the user is the behavior pattern of the user, and outputtinga warning.

In addition, the user is warned about a bad behavior even if the badbehavior is not the habit of the user. Accordingly, the presentdisclosure has an advantage of outputting a warning having a highintensity in case of a behavior originating from the habit of the userand outputting a warning having a low intensity in case of a badbehavior which is not the habit of the user, thereby weakly warning theuser about a bad behavior which is not a habit and warning the userabout a bad habit.

Although whether the notification is output or the intensity of thenotification is described as being determined or adjusted based onwhether the behavior of the user corresponds to the behavior pattern ofthe user in FIG. 6, the present disclosure is not limited thereto.

Specifically, the information on the behavior of the user is stored inthe personal database of the user as described above, and the artificialintelligence part may determine whether the notification is output oradjust the intensity of the notification based on whether the behaviorof the user corresponds to the behavior of the user included in thepersonal database.

For example, if the current behavior of the user is a behavior of notkeeping eyes forward, the standard behavior database includes a behaviorof not keeping eyes forward and the history of the user's behavior ofnot keeping eyes forward in the past is stored in the personal databaseof the user, the artificial intelligence part may output thenotification with a higher intensity.

In another example, if the current behavior of the user is a behavior ofnot keeping eyes forward, and the standard behavior database includes abehavior of not keeping eyes forward, but the history of the user'sbehavior of not keeping eyes forward in the past is not stored in thepersonal database of the user, the artificial intelligence part may notoutput the notification or may be output the notification with a lowerintensity.

FIG. 7 is a view illustrating a method of outputting a notificationusing a behavior pattern of a user corresponding to a standard behaviordatabase according to an embodiment of the present disclosure.

The method of operating the terminal 310 according to the embodiment ofthe present disclosure may include step S710 of acquiring the behaviorpattern of the user corresponding to the standard behavior databasebased on the behavior of the user and the standard behavior database andstoring the behavior pattern of the user in the personal database of theuser, step S720 of acquiring the information on the behavior of thebehavior of the user and step of outputting a notification when thebehavior of the user corresponds to the behavior pattern of the user.

Step S710 of acquiring the behavior pattern of the user corresponding tothe standard behavior database based on the behavior of the user and thestandard behavior database and storing the behavior pattern of the userin the personal database of the user will now be described.

The artificial intelligence part may acquire the behavior pattern of theuser corresponding to the standard behavior database based on thebehavior of the user and the standard behavior database.

Specifically, the artificial intelligence part may acquire theinformation on the behavior of the user and store the acquired behaviorof the user in the personal database of the user as the behavior patternof the user when the behavior of the user corresponds to the behavior ofthe driver included in the standard behavior database.

The behavior pattern may be acquired based on how often the specificbehavior of the user appears, the number of times that the specificbehavior of the user appears or a point in time when the specificpattern appears. How often the specific behavior of the user appears,the number of times that the specific behavior of the user appears orthe point in time when the specific pattern appears may be changedaccording to settings.

For example, even if the number of times that the user's behavior of notkeeping eyes forward appears is only one, the user's behavior of notkeeping eyes forward may be stored as the behavior pattern of the user.In another example, when the number of times that the user's behavior ofnot keeping eyes forward appears is equal to or greater than three, theuser's behavior of not keeping eyes forward may be stored as thebehavior pattern of the user. In addition, when the number of times thatthe user's behavior of not keeping eyes forward appears is equal to orgreater than three and is greater than one per month, the user'sbehavior of not keeping eyes forward may be stored as the behaviorpattern of the user.

The artificial intelligence part may acquire the information on thecurrent behavior of the user and output the notification when thecurrent behavior of the user corresponds to the behavior pattern storedin the personal database.

For example, when the behavior of not keeping eyes forward may be storedin the personal database of the user as the behavior pattern of the userand the current behavior of the user is the behavior of not keeping eyesforward, the artificial intelligence part may output the notification.

If the behavior of the user is compared with the standard behaviordatabase whenever the behavior of the user appears, the bad habit of theuser may be patterned and stored in advance. When the behavior of theuser appears later, since the process of comparing the behavior of theuser with the information patterned and stored in advance is onlyperformed, it is possible to more rapidly provide a guide to the user.

FIGS. 8 to 9 are views illustrating a method of outputting anotification based on an accident type database according to anembodiment of the present disclosure.

The accident type database may include information on a position andinformation on an accident type at the position. The information on theaccident type may be collected when a plurality of accidents occurs atthe position.

Specifically, an accident type database provider may acquire informationincluding the type of the accident when accidents occur at variouspositions. When such information is acquired from a plurality ofaccident cases, the accident type database provider may analyze thepatterns of the types of the accidents at various positions and generatethe accident type database including the information on the types of theaccidents at a plurality of positions.

For example, when a pattern in which accidents frequently occur at afirst position due to rear collision of a rear vehicle is derived fromthe plurality of accident cases, the accident type database may includethe first position and information on the rear collision at the firstposition.

In another example, when a pattern in which accidents frequently occurat a second position due to side collision while entering a left lane isderived from the plurality of accident cases, the accident type databasemay include the second position and information on left collision at thesecond position.

The controller may receive the accident type database from the accidenttype database provider and store the accident type database in thestorage unit.

FIG. 9 shows a method of building the accident type database.

Information included in the accident type database may be acquired byreceiving the images of an accident captured when the accident occursfrom an insurance company or a police officer and analyzing the images.

In addition, the information included in the accident type database maybe acquired by analyzing a document or data related to a trafficaccident situation.

Operation performed by the accident type database provider may beperformed by the artificial intelligence part of the terminal 310.

The method of operating the terminal 310 according to the embodiment ofthe present disclosure may include step S810 of acquiring information onthe position of the user and step S820 of outputting a notificationcorresponding to the accident type at the position when the user passesthe position included in the accident type database.

The artificial intelligence part may acquire information on the currentposition of the user based on the data collected by the driving statedetection device or the situation information acquisition unit.

In addition, the artificial intelligence part may output thenotification corresponding to the accident type at the position when theuser passes the position included in the accident type database.

For example, if a first position and information on right collision atthe first position are included in the accident type database, theartificial intelligence part may output a notification “Watch out foryour right” when the user passes the first position.

In another example, if a second position and information on signalviolation and speeding at the second position are included in theaccident type database, the artificial intelligence part may output anotification “Watch out for signal violation and speeding” when the userpasses the second position.

The user passing the position may include not only passing the positionbut also being going to pass the position.

FIG. 10 is a view illustrating a method of outputting a notificationcorresponding to an accident type based on an accident type database andinformation on a driving state of a user according to an embodiment ofthe present disclosure.

The method of operating the terminal 310 according to the embodiment ofthe present disclosure may include step S1010 of acquiring informationon the position of the user, step S1020 of acquiring information on thedriving state of the user, and step S1030 of outputting a notificationcorresponding to an accident type based on the driving state of the userand the accident type when the user passes a position included in theaccident type database.

For step S1010 of acquiring the information on the position of the user,refer to the description of FIG. 8.

Step S1020 of acquiring the information on the driving state of the userwill be described.

The artificial intelligence part may acquire the information on thedriving state of the user based on the data acquired by the drivingstate detection device 340 or the data acquired by the situationinformation acquisition unit of the terminal 310.

The information on the driving state may include a vehicle direction,angle, speed, acceleration, inclination, forward/reverse, leftturn/right turn, tire, vehicle lamp, lane, lane change information,neighboring vehicle information, etc.

Step of outputting the notification corresponding to the accident typebased on the driving state of the user and the accident type when theuser passes the position included in the accident type database will bedescribed.

When the user passes the position included in the accident typedatabase, the artificial intelligence part may output the notificationcorresponding to the accident type based on the driving state of theuser and the accident type.

For example, if a first position and information on speeding at thefirst position are included in the accident type database and the speedof the vehicle is equal to or greater than a regulation speed, theartificial intelligence part may output a notification “Watch out forspeeding” when the user passes the first position.

In addition, if a second position and information on left collision whenthe lane is changed from a third lane to a second lane at the secondposition are included in the accident type database and the vehicletravels in the third lane, the artificial intelligence part may output anotification “Watch out for your left”.

Although the notification corresponding to the accident type isdescribed as being output based on the driving state of the user and theaccident type, the present disclosure is not limited thereto and thenotification corresponding to the accident type may be output based onthe behavior of the user and the accident type.

For example, if a third position and information on the type of anaccident which changes a lane without turning on a turn signal at thethird position are included in the accident type database, theartificial intelligence part may output a notification “Turn on a turnsignal” based on information indicating that the user does not turn onthe turn signal and the accident type.

The present disclosure has an advantage of guiding the user into safedriving, by analyzing the type of the accident at a specific positionand outputting a notification according to the type of the accident.

In particular, by comparing the current driving state of the user withthe accident type database acquired from a plurality of cases, it ispossible to provide a more realistic guide.

FIG. 11 is a view illustrating a method of guiding a user based on adriving habit of a user according to an embodiment of the presentdisclosure.

The method of operating the terminal 310 according to the embodiment ofthe present disclosure may include step S1110 of acquiring a drivinghabit of a user, step S1120 of acquiring information on a position ofthe user and step S1130 of outputting a notification related to thedriving habit when the user reaches a position where the driving habitof the user appears.

Step S1110 of acquiring the driving habit of the user will be described.

The driving habit of the user is distinguished from the above-describedbehavior of the user and may mean a result of a behavior according touser's determination beyond a simple user's behavior.

Specifically, the driving habit may include traffic regulation violationinformation of the user.

The traffic regulation violation information of the user may be acquiredby the artificial intelligence part.

For example, the artificial intelligence part may acquire information onthe regulation speed of a road on which the user is located anddetermine whether the user violates the traffic regulations based on theregulation speed and the current speed of the user.

In another example, the artificial intelligence part may acquireinformation indicating that the user is turning left and informationindicating that the user has not turned on the turn signal, anddetermine that the user has violated the traffic regulations based onthe information indicating that the user is turning left and theinformation indicating that the user has not turned on the turn signal.

Although the traffic regulation violation information of the user isdescribed as being acquired by the artificial intelligence part, thepresent disclosure is not limited thereto and the traffic regulationviolation information of the user may be received from the trafficregulation violation information manager of a police server.

The driving habit may include route deviation information of the user.

The route deviation information may be acquired by the artificialintelligence part.

For example, when a specific route was set by navigation in the past andthe user deviates from the specific route and then enters another routeafter following the specific route, the artificial intelligence part maydetermine that the user has deviated from the route. In this case, theartificial intelligence part may store information on the specific routeand the other route.

In another example, when the navigation system performs route researchonce or plural times due to deviation from the route, the artificialintelligence part may determine that the user has deviated from theroute.

In addition, the route deviation information may be acquired by theartificial intelligence part outputting a question to the user.Specifically, when a specific route is set by the navigation system andthe user deviates from the specific route and enters another route afterfollowing the specific route, the artificial intelligence part mayoutput a question as to whether the user has intentionally orinadvertently deviated from the route.

The terminal 310 may include an input unit capable of receiving theanswer of the user. When an answer is received that the user hasintentionally deviated from the route, the artificial intelligence partmay store the other route in the personal database as the route, alongwhich the user has traveled. In contrast, when an answer is receivedthat the user has inadvertently deviated from the route, the artificialintelligence part may determine that the user has deviated from theroute.

Meanwhile, when the driving habit of the user is acquired, theartificial intelligence part may store the driving habit and positioninformation corresponding to the driving habit in the personal database.

The artificial intelligence part may acquire information on the currentposition of the user.

In addition, the artificial intelligence part may output a notificationrelated to the driving habit at a position where the driving habit ofthe user appears, based on the driving habit of the user and positioninformation corresponding to the driving habit.

Specifically, the artificial intelligence part may output a notificationfor preventing traffic regulation violation when reaching the positionwhere the traffic regulation violation appears, based on the trafficregulation violation information and the position informationcorresponding to traffic regulation violation.

For example, when a driving habit such as speeding is acquired andspeeding appears at a first position and a second position, theartificial intelligence part may output a notification “You speeded atthe current point last time. Watch out for speeding” when the userreaches the first position or the second position.

In addition, the artificial intelligence part may output a notificationfor preventing route deviation when the user reaches a position whereroute deviation appears, based on the route deviation information andposition information corresponding to route deviation.

For example, when the user deviates from a predetermined route at thefirst position, the artificial intelligence part may output anotification “You entered a road b without entering a road a at thispoint last time. Be careful”, when the user reaches the first position.

“Position where the driving habit appears” may mean not only a positionwhere the driving habit appears but also a position before reaching theposition where the driving having appears.

For example, when the user has deviated from the first position in thepast, the artificial intelligence part may determine that the user isgoing to the first position according to the route of the road and themovement direction of the user. In addition, the artificial intelligencepart may output a notification for preventing route deviation at apredetermined time before reaching the first position (or at apredetermined distance before the first position).

The user may not remember their traffic regulation violation or routedeviation and may repeat the same mistake. For example, the usermistaking a road and deviating from the route may repeatedly appear atthe same position.

The present disclosure has an advantage of reminding the user of pastmistakes by checking the driving habits of the user and outputting awarning using the same.

Although the notification for preventing traffic regulation violation isdescribed as being output when there was a traffic regulation violationin the above-described embodiment, the present disclosure is not limitedthereto.

Specifically, the artificial intelligence part may acquire a specificdriving habit, e.g., the driving habit of the user which may potentiallycause an accident. For example, the artificial intelligence part mayacquire the driving habit of the user which is not a traffic regulationviolation but may cause a dangerous situation. In addition, theartificial intelligence part may store a driving habit and positioninformation corresponding to the driving habit. In this case, theartificial intelligence part may output a notification related to thedriving habit at a position where the driving habit appears. Forexample, the artificial intelligence part may output a notification“There is a pedestrian crossing as soon as you cross this hill, sospeeding on the hill like last time may be dangerous.

The artificial intelligence part may acquire a driving habit includingthe speed pattern of the user. In this case, the artificial intelligencepart may output a notification for inducing the user to drive at aneconomic speed based on the driving habit of the user and the economicspeed.

When the user uses a route used in the past again and the road conditionof the used route has been changed, the artificial intelligence part mayoutput a warning alarm. For example, the artificial intelligence partmay output a notification “The speed limit of the road has been changedfrom 80 to 50 as compared to when you came last time.

The artificial intelligence part may output a notification to check tirepressure when the weather suddenly changes.

In addition, based on a previous vehicle checking timing, the artificialintelligence part may recommend a next vehicle checking timing.

The terminal 310 may output a notification based on who the user is.

Specifically, the storage unit may include a plurality of personaldatabases respectively corresponding to a plurality of users. Forexample, the behavior, behavior pattern and driving habit of a firstuser are stored in a first personal database corresponding to the firstuser and the behavior, behavior pattern and driving habit of a seconduser are stored in a second personal database corresponding to thesecond user.

In addition, when the user sits on a driver's seat, the artificialintelligence part of the terminal 310 may determine who the user sittingon the driver's seat is, output a notification based on a personaldatabase corresponding to the user sitting on the driver's seat, andstore information acquired while driving in the personal databasecorresponding to the user sitting on the driver's seat.

Behaviors or habits which occur while driving may differ between users.The present disclosure has an advantage of providing a driver-specificguide, by building a personal database of each user and guiding eachuser.

In addition, the artificial intelligence part of the terminal 310 maycontrol the display position of an image based on who the user sittingon the driver's seat is.

Specifically, the artificial intelligence part of the terminal 310 maydisplay an image at a position optimized for each user, which differsbetween users, based on the behavior pattern of the user.

For example, if the behavior pattern of a first user frequently sees aright 15-degree point, the artificial intelligence part of the terminal310 may control the output device 350 such that an image is projectedonto the right 15-degree point of the first user.

In another example, if the field of view of a second user is from left30 degrees to right 30 degrees, the artificial intelligence part of theterminal 310 may control the output device 350 such that an image isprojected within the field of view of the second user. In this case, theartificial intelligence part may change the display position of theimage based on the field of view and movement speed of the user. Forexample, if the movement speed is high, since the field of view becomesnarrower, the artificial intelligence part may display the image withina narrower field of view.

In another example, if a third user sees a left sideview mirror moreoften than a right sideview mirror, the artificial intelligence part maydisplay major information of the vehicle on the right side of thescreen.

The artificial intelligence part of the terminal 310 may transmit acontrol signal to a device capable of changing a sideview mirror, arearview mirror, a seat, a song playlist, a recent destination fornavigation, such that the driving environment corresponding to the usersitting on the driver's seat is created based on who the user sitting onthe driver's seat is.

The artificial intelligence part of the terminal 310 may storeconversation or saying goodbye when each user gets off the vehicle andoutput saying hello when the user rides in the vehicle.

In addition, the artificial intelligence part of the terminal 310 mayoutput a preferred voice of each driver, such that the driver does notfeel uncomfortable.

In addition, the artificial intelligence part may control a vehicleinside-temperature control system to maintain a preferred temperature ofeach driver.

In addition, the artificial intelligence part may provide the navigationhistory and recent destination to each driver.

Although the terminal is described as being the terminal 310 configuringthe driver guide system 300 in FIGS. 2 to 11, the mobile terminal 100described with reference to FIG. 1 will now be described.

FIGS. 12 to 15 are views illustrating a method of guiding a route basedon a route pattern of a user according to an embodiment of the presentdisclosure.

The method of operating the terminal 310 according to the embodiment ofthe present disclosure may include step S1210 of storing the route,along which the user has moved, step S1220 of acquiring information on amovement route to be used by the user based on a past movement route ofthe user and current movement of the user, and step S1230 of providing aguide to a new route based on the route information of the routeinformation to be used by the user.

Step S1210 of storing the route, along which the user has moved, will bedescribed.

The artificial intelligence part 130 of the mobile terminal 100 maystore information on the route, along which the user has moved.

Specifically, the artificial intelligence part 130 may store theinformation on the route, along which the user has moved, in thepersonal database of the user based on position information (e.g., GPS)acquired by the wireless communication unit 110.

In this case, the artificial intelligence part 130 may acquire theinformation on the route, along which the user has moved, byinterworking with a route finding application as shown in FIG. 13 or byinterworking with a navigation application as shown in FIG. 14.

Specifically, when a navigation system has guided the user to a specificroute and the user has moved along the specific route, the artificialintelligence part 130 may determine the specific route as the route,along which the user has moved.

In addition, when a navigation system has guided the user to a specificroute and the user has moved to a route other than the specific route,the artificial intelligence part 130 may determine the other route asthe route, along which the user has moved. In this case, when the sameuser sitting on the driver's seat again, the artificial intelligencepart 130 may guide the user to the other route.

The artificial intelligence part may acquire information on a movementmethod of the user based on the speed of the user, a movement route,movement of user such as walking or stopping, etc. In this case, theartificial intelligence part may store the movement method along withthe route, along which the user has moved.

For example, if the movement route of the user matches the route of thesubway line 5 and the user is moving but steps of the user are notdetected, the artificial intelligence part may determine that the useris traveling on the subway.

In another example, when the user is moving at a speed of 5 km/h or lessand user's steps are detected, the artificial intelligence part maydetermine that the user is walking.

In another example, when the user is moving at a speed of 80 km/h ormore and user's steps are not detected, the artificial intelligence partmay determine that the user is traveling using a vehicle.

In another example, when the route of the user matches the route of abus 100 and user's steps are detected and then are not detected from abus stop of the bus 100, the artificial intelligence part may determinethat the user is traveling on the bus.

Step S1220 of acquiring the information on the movement route to be usedby the user based on the past movement route of the user and the currentmovement of the user will be described.

The artificial intelligence part 130 may acquire the information on themovement route to be used by the user based on the past movement routeof the user and the current movement of the user.

For example, when the user has moved along a first route and a secondroute in the past and the user is currently moving along the firstroute, the artificial intelligence part may determine that the user willuse the first route and then use the second route.

In addition, the artificial intelligence part 130 may acquireinformation on the current movement method of the user based on thespeed of the user, the movement route and the movement of the user suchas walking or stopping. In addition, the artificial intelligence part130 may acquire the information on the movement route to be used by theuser based on the current movement method of the user, the currentmovement of the user, the past movement route of the user and themovement method of the past movement route of the user.

For example, if the user has traveled on the subway along a first routeand a second route in the past, has moved on a vehicle along the firstroute and a third route and the user is currently traveling on thesubway along the first route, the artificial intelligence part maydetermine that the user will use the second route after using the firstroute.

In addition, the artificial intelligence part 130 may store informationon the departure point and destination of the route, along which theuser has moved. In addition, the artificial intelligence part 130 maystore information on a time required for the user to move along theroute.

In this case, the artificial intelligence part 130 may provide a guideto a new route based on the rote information of the movement route to beused by the user.

The route information may include the traffic condition of the route.

For example, when the user is stuck in a traffic jam due to a caraccident occurring on the movement route to be used by the user, theartificial intelligence part 130 may output a guide “A traffic jam hasoccurred due to a car accident occurring on Road 3, so please use Road2”.

In this case, the artificial intelligence part may receive real-timetraffic condition information shown in FIG. 15 from the outside.

Meanwhile, the route information may include a route faster than a routeused by the user in the past.

Specifically, if the route used by the user in the past was not thefastest route to a destination, the artificial intelligence part mayoutput the fastest route to the destination.

For example, the route, along which the user has moved in the past, is afirst route, a second route and a third route but the fastest route tothe destination is the first route, a fourth route and the third route,the artificial intelligence part may output the first route, the fourthroute and the third route, which are faster than the route, along whichthe user has moved in the past.

The artificial intelligence part 130 may provide a guide to a new routebased on route information corresponding to the movement route to beused by the user and the movement method.

For example, when the user is current traveling using a vehicle and theroute to be used is congested, the artificial intelligence part 130 mayprovide a guide to a new route, along which the vehicle may travel.

In another example, when the user is currently traveling using a bus anda route, along which the user will move, is congested, the artificialintelligence part 130 may provide a guide to a route to transfer to asubway and get to a destination.

In another example, although the user has traveled in the firstcompartment of the subway in the past, if getting on the lastcompartment of the subway is faster in order to transfer the subway, theartificial intelligence part 130 may output a notification indicatingthat getting on the last compartment of the subway is faster.

In addition, the artificial intelligence part 130 may output informationon an estimated time required to move along the movement route to beused by the user based on the route information.

In addition, the artificial intelligence part 130 may output a result ofcomparing a time required for the user to move along the route in thepast with an estimated time required to move along the movement route tobe used by the user. For example, the artificial intelligence part 130may output a message “You will arrive 10 minutes later than yesterday”.

Although the movement route to be used by the user is described as beingdetermined based on the route, along which the user moved in the past,in FIGS. 12 to 15, the present disclosure is not limited thereto.

Specifically, the artificial intelligence part 130 may acquire the routepattern of the user based on how often the user uses the route, thenumber of times that the user uses the route, a time when the user usesthe route, etc. with respect to the routes, along which the usertraveled in the past.

For example, when the user has traveled along a specific route once ormore a week and has used the specific route five times or more, theartificial intelligence part may determine the specific route as theroute pattern of the user and store the determined route pattern in thepersonal database.

In this case, the artificial intelligence part 130 may acquireinformation on the movement route to be used by the user based on theroute pattern of the user and the current movement of the user.

People may maintain the same living habit. For example, people go towork using the same way to work. In this case, in the presentdisclosure, it is possible to learn a route which is normally used andrecommend a faster route in consideration of the traffic condition ofthe route used by the user or a problem of the route used by the user.

The artificial intelligence part 130 may acquire information on thesituation of the user.

Specifically, the artificial intelligence part 130 may acquireinformation on the situation of the user based on the situationinformation acquired by the wireless communication unit 110, the sensingunit 140 or the input unit 120, such as the position of the user, thereceived wireless signal, the speed of the user, the movement route,movement of the user such as walking or stopping, etc.

For example, the artificial intelligence part 130 may determine whetherthe user rides in a vehicle, a bus or a subway based on the movementroute, speed or movement of the user.

In addition, the artificial intelligence part 130 may determine whetherthe user does grocery shopping or is near a place capable of doinggrocery shopping, based on the situation information such as theposition of the user, the received wireless signal or the movement ofthe user.

In addition, the artificial intelligence part 130 may determine whetherthe user is at home or at work based on the situation information suchas the position of the user, the received wireless signal or themovement of the user.

In this case, the artificial intelligence part 130 may output thenotification based on the situation of the user.

For example, when the user is riding in a car, the artificialintelligence part 130 may output the notification described withreference to FIGS. 3 to 11.

In another example, as described with reference to FIG. 12, theartificial intelligence part 130 may output a notification correspondingto the situation of the user based on the current situation of the user(that is, the movement method of the user).

In another example, if the user does grocery shopping or is near a placecapable of doing grocery shopping, the artificial intelligence part 130may output a notification indicating the consumption state of the useror a notification for purchasing an item.

In another example, when the user is at home, the artificialintelligence part 130 may recommend a life pattern recommended to theuser.

FIGS. 16 to 17 are views illustrating a method of outputting anotification based on a consumption pattern of a user according to anembodiment of the present disclosure.

Referring to FIG. 16, the method of operating the mobile terminal 100according to the embodiment of the present disclosure may include stepS1610 of acquiring the consumption pattern of a user based on theconsumption list of the user and step S1620 of outputting a notificationindicating the consumption state of the user based on the consumptionpattern and the consumption list.

The personal database of the user may include the consumption list ofthe user.

Specifically, the artificial intelligence part 130 may acquire and storethe consumption list of the user in the personal database of the user.

The consumption lit may include at least one of a purchased item, apurchase price, a payment method or a payment time. The payment methodmay include micro payment, card payment, account transfer, etc.

The artificial intelligence part 130 may acquire the consumption list ofthe user based on payment information.

For example, the artificial intelligence part 130 may acquire theconsumption list of the user using a text message related to micropayment.

In another example, the artificial intelligence part 130 may acquire theconsumption list of the user using a payment-related text messagereceived from a card company, as shown in FIG. 17.

In another example, the artificial intelligence part 130 may acquire theconsumption list of the user using information included in an accounttransfer window.

The artificial intelligence part 130 may acquire the consumption patternof the user based on the consumption list of the user.

Specifically, the artificial intelligence part 130 may acquire theconsumption pattern of the user based on at least one of how often aspecific consumption appears, the number of times that the specificconsumption of the user appears or a time when the specific consumptionappears. In addition, the artificial intelligence part 130 may store theacquired consumption pattern in the personal database of the user.

For example, the artificial intelligence part 130 may store consumptionof paying for a game item as the consumption pattern of the user, whenconsumption of paying for the game item appears once or more a week.

In another example, the artificial intelligence part 130 may storeconsumption of making payment using a micro payment method as theconsumption pattern of the user, when consumption of making paymentusing the micro payment method appears 10 times or more.

In addition, the artificial intelligence part 130 may store the amountof money consumed during a predetermined period as the consumptionpattern of the user based on the purchase price. For example, theartificial intelligence part 130 may store information indicating thatthe user spends an average of 800,000 won per month as the consumptionpattern.

The artificial intelligence part 130 may output a notificationindicating the consumption state of the user based on the consumptionpattern and consumption list of the user.

For example, if the consumption pattern of the user is micro payment andthe micro payment is primarily made at the beginning of a month, theartificial intelligence part 130 may output a notification including amessage “You tends to make micro payment at the beginning of a month”.

In another example, based on a consumption pattern in which the userspends an average of 800,000 won per month and a consumption patternindicating that 400,000 won is spent up to now on 10^(th) day of thismonth, the artificial intelligence part 130 may output a notificationincluding a message “50% of money spent on the previous month is spent,so you will be in trouble at the end of the month”.

The artificial intelligence part 130 may acquire information on an itemto be purchased by the user and output a notification indicating theconsumption state of the user based on the item to be purchased by theuser and the consumption list.

For example, when an image or text related to a lipstick is displayed onthe display 151 or a payment site (or a payment window) related to thelipstick is displayed on the display 151, the artificial intelligencepart 130 may determine that an item to be purchased by the user is alipstick. In addition, when a lipstick purchased last week is includedin the consumption list, the artificial intelligence part 130 may outputa notification “You purchased a lipstick last week”.

In the present disclosure, it is possible to induce healthy consumptionof the user, by analyzing the consumption pattern of the user and givinga warning.

FIG. 18 is a view illustrating a method of providing a guide to an itemto be purchased according to an embodiment of the present disclosure.

The method of operating the mobile terminal 100 according to theembodiment of the present disclosure may include step S1810 of acquiringinformation on an item which needs to be purchased, step S1820 ofacquiring information on the situation of the user, and step S1830 ofoutputting a notification for purchasing an item in the vicinity of aplace where the item which needs to be purchased is capable of beingacquired.

Step S1810 of acquiring information on the item which needs to bepurchased will now be described.

The personal database may include a list of items.

The list of items may be a list of items purchased by the user or a listof items in the house of the user. The list of items may include apurchased item, a purchase price, a payment method, a purchase time, adistribution period, whether the item is used, a residual ratio of theitem, or a purchase frequency.

The list of items may be acquired based on payment information. Inaddition, the list of items may be acquired based on an image of apayment document such as a receipt.

The artificial intelligence part 130 may acquire information on the itemwhich needs to be purchased based on at least one of the purchase time,purchase frequency or distribution period of an item included in thelist of items.

For example, when milk was purchased 15 days ago and the distributionperiod of milk is 14 days, the artificial intelligence part 130 maydetermine that milk needs to be purchased.

In another example, when a consumption pattern in which the userpurchases milk once every two weeks and the user purchased milk 17 daysago, the artificial intelligence part 130 may determine that milk needsto be purchased.

The artificial intelligence part 130 may acquire information on thesituation of the user.

For example, when the user repeatedly walks and stops and is located ata place where a store is located, the artificial intelligence part 130may acquire information indicating that the user is doing groceryshopping.

In another example, when the user is near a store, the artificialintelligence part 130 may acquire information indicating that the usermay purchase an item.

In another example, when the user is near a store but is driving, theartificial intelligence part 130 may acquire information indicating thatthe user cannot purchase an item.

The artificial intelligence part 130 may output a notification forpurchasing an item based on information on the situation of the user.

For example, if the item which needs to be purchased is milk, theartificial intelligence part 130 may receive position information of astore which is selling milk. When the user is near the store, theartificial intelligence part 130 may output a message “Milk waspurchased 15 days ago, so milk needs to be purchased”.

In another example, the artificial intelligence part 130 may receive theposition information of a place where milk is located in the store. Inaddition, when the user in the store passes the place where milk islocated, the artificial intelligence part 130 may output a message “Milkwas purchased 15 day ago and the distribution period of milk is 16days”.

In another example, when the user in the store passes a place wheretoilet paper is located, the artificial intelligence part 130 mayreceive a wireless signal from the place where toilet paper is located.In addition, among the items in the user's house, there is only littletoilet paper. In this case, the artificial intelligence part 130 mayoutput a message “You purchased the toilet paper next to you on 5^(th)day of last month and the toilet paper is likely to be be used up afterabout ten days”

The present disclosure has an advantage of determining the currentsituation of the user and appropriately providing the informationrequired by the user in the current situation. Specifically, byproviding information on a necessary item based on a purchase list and aconsumption pattern, it is possible to induce the necessary item to bepurchased in an optimal state.

The artificial intelligence part 130 may recommend an item which needsto be purchased based on the situation of the user. For example, whenthe user is driving, the artificial intelligence part 130 may recommendpurchase of an item which needs to be purchased in association with thevehicle.

FIG. 19 is a view illustrating a method of outputting a life patternrecommended to a user based on information on a preferable life patternof a person according to an embodiment of the present disclosure.

The method of operating the terminal according to the embodiment of thepresent disclosure may include step S1910 of acquiring and storing alife pattern of a user in a personal database, step S1930 of acquiringinformation on the behavior of the user, step S1930 of determiningwhether the behavior of the user corresponds to the behavior of a personof the standard behavior database, step of S1940 of determining whetherthe behavior of the user corresponds to the behavior pattern of the userwhen the behavior of the user corresponds to the behavior of the personof the standard behavior database, and step S1950 of determining whethera notification is output or adjusting the intensity of the notificationbased on whether the behavior of the user corresponds to the behaviorpattern of the user.

The database may include the personal database of the user.

Step S1910 of acquiring and storing the life pattern of the user in thepersonal database will now be described.

The information on the situation of the user may be stored in thepersonal database of the user. Specifically, the artificial intelligencepart 130 may store the information on the behavior of the user in thepersonal database of the user.

For example, the artificial intelligence part 130 may store informationon a time when the user returns home based on the position informationof the mobile terminal 100. In addition, the artificial intelligencepart 130 may store information on when use of the mobile terminal 100 isterminated or information on a time when use of the mobile terminal 100is restarted, based on a call duration of the mobile terminal 100, anapplication usage time and movement of the mobile terminal 100.

The personal database of the user may include information on the lifepattern of the user. The life pattern of the user may be acquired basedon a plurality of past behaviors of the user.

Specifically, the artificial intelligence part may acquire the lifepattern of the user, based on at least one of how often a specificbehavior of the user appears, the number of times that the specificbehavior of the user appears or a point in time when the specificpattern appears.

For example, if the mobile terminal 100 is used until late everyThursday, the artificial intelligence part 130 may determine that theuser goes to sleep late on Thursday.

In another example, if the user makes a telephone call every Mondaymorning, the artificial intelligence part 130 may determine that theuser makes many telephone calls every Monday morning.

In addition, the artificial intelligence part 130 may acquire a lifepattern including the sleep time, wakeup time, sleep time of the userbased on a time when the mobile terminal 100 is used or movement of themobile terminal 100.

The artificial intelligence part 130 may acquire information on thecurrent behavior of the user.

In addition, the artificial intelligence part 130 may determine whetherthe current behavior of the user corresponds to the life pattern of theperson of the standard behavior database.

Information on the life pattern of the person is based on the WHO'srecommended guideline and may include information on a recommended sleeptime, a recommended bedtime, a recommended wakeup time, etc.

For example, if the user is still using the mobile terminal 100 at 2:00am and the standard behavior database includes information indicatingthat the recommended bedtime is 11:00, the artificial intelligence part130 may determine that the behavior of the user does not conform to thelife pattern of the person.

In another example, if the user has used the mobile terminal 100 until2:00 am and has used the mobile terminal 100 again at 7:00 am, and thestandard behavior database includes information indicating that therecommended sleep time is eight hours, the artificial intelligence part130 may determine that the behavior of the user does not conform to thelife pattern of the person of the standard database.

If the behavior of the user does not conform to the life pattern of theperson of the standard database, the artificial intelligence part 130may output a warning notification. In addition, the artificialintelligence part 130 may output the recommended life pattern to theuser.

For example, the artificial intelligence part 130 may output a messagesaying “Why don't you reduce the number of telephone calls a littleevery Monday morning?”.

If the current behavior of the person does not conform to the lifepattern of the person stored in the standard behavior database, theartificial intelligence part 130 may determine whether the currentbehavior of the user is stored in the personal database as the lifepattern of the user. In this case, the artificial intelligence part 130may determine whether the notification is output based on whether thecurrent behavior of the user corresponds to the life pattern of theuser.

For example, if the user goes to sleep late only once and is not thenormal life pattern of the user, the artificial intelligence part 130may not output the notification.

In another example, if it is Thursday and the user going to sleep lateon Thursday is the life pattern of the user, the artificial intelligencepart 130 may output a notification “Your bedtime is usually too late onThursday. Go to bed earlier”.

Based on whether the behavior of the user corresponds to the lifepattern of the user, the artificial intelligence part 130 may adjust theintensity of the notification.

For example, if the user does not go to sleep until late and this is notthe life pattern of the user, the artificial intelligence part 130 mayoutput the notification with a low intensity.

In another example, if the user does not go to sleep until late and thisis the life pattern of the user, the artificial intelligence part 130may output the notification with a high intensity.

The notification with the low intensity and the notification with thehigh intensity may be distinguished by the level, the frequency and thelike.

Outputting a notification whenever a specific life pattern of a userappears may cause fatigue to the user. Accordingly, the presentdisclosure has an advantage of outputting a warning only with respect tothe bad habits of the user, by extracting the life pattern of the userfrom the behaviors of the user, determining whether a specific behaviorof the user is the life pattern of the user, and outputting a warning.

In addition, the user is preferably warned about a bad behavior even ifthe bad behavior is not the habit of the user. Accordingly, the presentdisclosure has an advantage of outputting a warning having a highintensity in case of a behavior originating from the habit of the userand outputting a warning having a low intensity in case of a badbehavior which is not the habit of the user, thereby weakly warning theuser about a bad behavior which is not a habit and warning the userabout a bad habit.

Although whether the notification is output or the intensity of thenotification is described as being determined or adjusted based onwhether the behavior of the user corresponds to the behavior pattern ofthe user in FIG. 19, the present disclosure is not limited thereto.

Specifically, the information on the behavior of the user is stored inthe personal database of the user as described above, and the artificialintelligence part may determine whether the notification is output oradjust the intensity of the notification based on whether the situationof the user corresponds to the behavior of the user included in thepersonal database.

Meanwhile, the controller generally controls the device and may be usedinterchangeably with a central processing unit, a microprocessor, aprocessor, etc.

The aforementioned present disclosure can also be embodied as computerreadable code stored on a computer readable recording medium. Thecomputer readable recording medium is any data storage device that canstore data which can thereafter be read by a computer. Examples of thecomputer readable recording medium include a hard disk drive (HDD), asolid state drive (SSD), a silicon disk drive (SDD), read-only memory(ROM), random-access memory (RAM), CD-ROM, magnetic tapes, floppy disks,optical data storage devices, etc. The computer may also include theprocessor 180 of a terminal. It will be apparent to those skilled in theart that various modifications and variations can be made in the presentdisclosure without departing from the spirit or scope of the disclosure.Thus, it is intended that the present disclosure cover the modificationsand variations of this disclosure provided they come within the scope ofthe appended claims and their equivalents.

1. A terminal comprising: an output interface configured to output anotification; a memory configured to store a database; a controllerconfigured to control output of the notification; and an artificialintelligence interface configured to acquire information on a situationof a user and output a notification when the situation of the usercorresponds to information included in the database, wherein thedatabase includes at least one of a personal database of the user, astandard behavior database or an accident type database.
 2. The terminalof claim 1, wherein the standard behavior database includes informationon a behavior of a driver while driving, and wherein the artificialintelligence interface acquires information on a behavior of the userand outputs the notification when the behavior of the user correspondsto the behavior of the driver included in the standard behaviordatabase.
 3. The terminal of claim 2, wherein the standard behaviordatabase includes information on a position and information on abehavior of a driver at the position, and wherein the artificialintelligence interface outputs the notification when the user passes theposition and the behavior of the user corresponds to the behavior of thedriver at the position.
 4. The terminal of claim 2, wherein the standardbehavior database includes information on an incorrect behavior of adriver while driving, wherein the memory includes information on acorrect behavior corresponding to the incorrect behavior, and whereinthe artificial intelligence interface outputs the correct behavior whenthe behavior of the user corresponds to the incorrect behavior.
 5. Theterminal of claim 2, wherein the personal database of the user includesinformation on a behavior pattern acquired from a past behavior of theuser, and wherein the artificial intelligence interface acquiresinformation on the behavior of the user and determines whether thenotification is output or an intensity of the notification based onwhether the behavior of the user corresponds to the behavior pattern. 6.The terminal of claim 1, wherein the artificial intelligence interfaceacquires and stores a behavior pattern of a user corresponding to thestandard behavior database based on a behavior of the user and thestandard behavior database, and outputs the notification when thebehavior of the user corresponds to the behavior pattern.
 7. Theterminal of claim 1, wherein the accident type database includesinformation on a position and information on an accident type at theposition, and wherein the artificial intelligence interface outputs anotification corresponding to the accident type at the position when theuser passes the position.
 8. The terminal of claim 7, wherein theartificial intelligence interface acquires information on a drivingstate of the user, and outputs a notification corresponding to theaccident type based on the driving state of the user and the accidenttype when the user passes the position.
 9. The terminal of claim 1,wherein the artificial intelligence interface acquires a driving habitincluding at least one of information on a traffic regulation violationof the user or information on route deviation of the user, stores thedriving habit and position information corresponding to the drivinghabit in the personal database, and outputs a notification related tothe driving habit at a position where the driving habit appears.
 10. Theterminal of claim 9, wherein the artificial intelligence interfaceoutputs a notification for preventing the traffic regulation violationwhen the user reaches a position where the traffic regulation violationappears, based on the information on the traffic regulation violationand position information corresponding to the traffic regulationviolation.
 11. The terminal of claim 9, wherein the artificialintelligence interface outputs a notification for preventing the routedeviation when the user reaches a position where the route deviationappears, based on the information on the route deviation and positioninformation corresponding to the information on the route deviation. 12.The terminal of claim 9, wherein the artificial intelligence interfaceacquires information on a movement route to be used by the user based ona past movement route of the user and current movement of the user andprovides a guide to a new route based on route information of themovement route to be used by the user, and wherein the route informationincludes at least one of a traffic condition of the route or a routefaster than a route used by the user in the past.
 13. The terminal ofclaim 1, wherein the personal database includes a consumption list ofthe user, and wherein the artificial intelligence interface acquires aconsumption pattern of the user based on the consumption list of theuser, and outputs a notification indicating a consumption state of theuser based on the consumption pattern and the consumption list.
 14. Theterminal of claim 1, wherein the personal database includes a list ofitems of the user, and wherein the artificial intelligence interfaceacquires information on an item, which needs to be purchased, based onat least one of a purchase time, a purchase frequency or a distributionperiod of an item included in the list of items, and outputs anotification for purchasing the item in the vicinity of a place wherethe item, which needs to be purchased, is capable of being acquired. 15.The terminal of claim 1, wherein the standard behavior database includesinformation on a person's life pattern, and wherein the artificialintelligence interface acquires information on a life pattern of theuser, and outputs a warning notification based on information on thelife pattern of the user and information on the person's life pattern.